gmane.comp.python.buildbot.devel

Re: Conceptual Questions: Change-based BuildStep

Kenny Ho — 2013-05-19T18:55:35

On Sun, May 19, 2013 at 2:35 PM, Kenny Ho <kenny.ho-idok+zigH+7QT0dZR+AlfA< at >public.gmane.org> wrote:
  path 2 ... to Change N where N is the total number of ChangeS in the Build.  Is this a supported use case?

I can't update the repos because these changes are actually not merged yet ("patchset-created" in gerrit terms) and can be rejected pending on the result of code review.  I am trying to replicate Pierre Tardy's work back from 2011/2012 (http://video.linux.com/videos/buildbot-and-gerrit-integration-improved-ci-automation) to automate building/testing of patches before they are merged.

Kenny
------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d

Re: Conceptual Questions: Change-based BuildStep

Dustin J. Mitchell — 2013-05-19T18:37:08

 path 2 ... to Change N where N is the total number of ChangeS in the Build.  Is this a supported use case?

I don't see why you'd need to cherry-pick there.  Couldn't you update
the relevant repos?

It might be best to model each git repository in the gerrit project as
a codebase.

Dustin

------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d

Re: Conceptual Questions: Change-based BuildStep

Kenny Ho — 2013-05-19T18:35:27

Hi Dustin,


Um... essentially.  The changes are not from the same codebase or respository though (I am not sure if I am using the term correctly), otherwise I can probably take advantage of the source stability timer and only build the newer head.  I am dealing with Gerrit which is a collection of git repositories.  There are times that I may have changes in the same gerrit project but on different git repositories that may be related.  I have configured the source stability timer to generate only one Build/BuildRequest when this situation happens and I can see that it works (the Build web UI showed that there are multiple ChangeS in one build, and listed them out correctly.)  I am hoping that there is some way for me to put in the BuildStep to say... git cherry pick Change 1 to path 1, Change 2 to pa
 th 2 ... to Change N where N is the total number of ChangeS in the Build.  Is this a supported use case?

Regards,
Kenny

------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d

Re: Conceptual Questions: Change-based BuildStep

Dustin J. Mitchell — 2013-05-19T18:14:28

Changes are really only messages to say "hey, the source code
changed".  Your build can get multiple *sourcestamps*, one for each
codebase, but each sourcestamp corresponds to one change.

It sounds like you're interested in a case where each change is a new
head (in hg terms), and when they occur close in time, you want to
cherry-pick all of them together and build the result.  Is that about
right?

Dustin

------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d

Re: User Question: 1 builder -> 2 buildslaves

sergio borghese — 2013-05-18T16:54:09

Hi All,

just tried Ludovic & Mattias suggestion sucessfully, thanks to both of you
for the quick reply.
I was misunderstanding the 1-builder : N-slaves use case model
As Ludovic explaind, it is more for load balancing, rather than
distributing on different slaves.
My buildmaster cfg file is rather structured, so I found quite easy to add
an extra builder :)

Many Thanks
Sergio


On Sat, May 18, 2013 at 6:44 PM, Ludovic Chabant <ludovic-+coQhnsWSdJBDgjK7y7TUQ< at >public.gmane.org>wrote:

Re: User Question: 1 builder -> 2 buildslaves

Ludovic Chabant — 2013-05-18T16:44:54

On Sat, May 18, 2013 at 8:24 AM, sergio borghese
<sergio.borghese-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org>wrote:


Yes, you just declare what builders *can* run on what buildslaves, so that
buildbot can do a bit of logic for distributing builds across buildslaves
when lots of events come in (load balancing, etc).

If you have 2 builders that are not completely equivalent (they do the same
thing but must run on different OSes), the easiest way to handle that would
be to, in fact, create 2 builders. Because the master.cfg file is a regular
Python file, you can have a bit of logic to generate the same builder
instance or builder factory twice. Then you can link one builder to one
slave, and the other to the other slave. Then you trigger both builders
from your scheduler.

--
   l u d o .
   . 8 0 17 80
------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d_______________________________________________
Buildbot-devel mailing list
Buildbot-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
https://lists.sourceforge.net/lists/listinfo/buildbot-devel

User Question: 1 builder -> 2 buildslaves

sergio borghese — 2013-05-18T15:24:15

Hi All,

I have a question I cannot find an answer after reading the manual, and
doing some test; hope someone can help here;

I setup a buildmaster and two buildslaves on two distinct hosts (Ubuntu
12.04 and Debian Weezy)
Configured a builder and assigned to both slaves to build my code on both
the hosts
When the SVNPoller notice something has changed in the repo, I was
expecting the build process to start on both hosts, to test the build on
both O.S.
It seams the build process starts only on one of the slaves instead, and
the one that is started seem random (I did not see a pattern)
I notice there is a param "nextSlave" to decide which slave to run. So my
question is: If I set multiple slaves for the same builder, only one slave
is run?
Can I set up the master/slave system to kick the build on all my
buildslaves at once?

Many thanks
Sergio

Getting pass/fail status from JSON API

Michael Williamson — 2013-05-18T10:12:41

Hello,

Is there an easy way to get the pass/fail status of a builder (or all
builders) from the JSON API? The examples I've seen seem to look for
particular text substrings in the "text" property, which seems a little
brittle.

Thanks

Mike

------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d

Conceptual Questions: Change-based BuildStep

Kenny Ho — 2013-05-16T21:24:05

Hi,

I am still fairly new to BuildBot and I am still a bit confused about some of the concepts.  Is it possible to define a BuildStep via BuildFactory with information from multiple changes?  For example, let say one of the Build contain a SourceStamp that points to 2 ChangeS.  Is it possible to define a BuildStep to cherry pick those 2 ChangeS to current HEAD?   or do I need to subclass something to archive this?

I noticed there are Properties set but they appear to belong to only one of the change?  (or may be it's just the web UI showing properties of one change and there's actually an array of build properties?)

Regards,
Kenny
------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d_______________________________________________
Buildbot-devel mailing list
Buildbot-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
https://lists.sourceforge.net/lists/listinfo/buildbot-devel

e-mail notifications based on scheduler type

Markos Chandras — 2013-05-16T19:28:10

Hi everybody,

I have e-mail notifications configured for certain builders. However,
these builders are triggered either by different schedulers
(Triggering, Try, Force etc). Is it possible to send e-mail
notifications for a builder but only if it was triggered by a
particular scheduler? I see no such option in the MailNotifier class
so I was wondering if there is a "good" way to do that.

--
Regards,
Markos Chandras

------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d

Re: Template folder

Dustin J. Mitchell — 2013-05-15T21:55:24

If you're running Buildbot-0.8.x, then you can do that by putting your
templates in your master's basedir.  See
http://docs.buildbot.net/latest/manual/cfg-statustargets.html#configuration

Things are quite a bit different in nine -- the UI has, indeed, moved,
and is now implemented in JS (well, CoffeeScript, close enough).  If
you're interested in working on that project, we'd love the help!

Dustin

------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d

Re: Tools for GSoC mentors

Flower Platform Team — 2013-05-15T19:29:59

Hello Brad,

1) Right now Flower Dev Center is closed source, and offered for free to 
every open source project (not just related to GSoC).
2) After the 2.0.0 release planned in June/July, we'll begin to open 
it's API and source code, so that everyone can add modules and 
functionality.

Best regards,
Cristian.


On 15.05.2013 18:07, Brad Hards wrote:


------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d

Re: Tools for GSoC mentors

Brad Hards — 2013-05-15T16:07:46

Just to confirm: this is a closed source tool, but you are offering it 
for "free-as-in-beer" use for students in GSoC?

Brad



------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d

Tools for GSoC mentors

Flower Platform Team — 2013-05-15T14:28:08

Hello GSoC Mentors,

As you are participating to Google Summer of Code, I'm wondering if you
could consider using Flower Dev Center [1] while working with students.

Flower Dev Center is an online platform for UML modeling diagramming,
with a strong focus on code synchronization, integration with dev tools
(Git, SVN, etc) and real time collaboration on diagrams (and a little
bit on code as well).

We think Flower Dev Center can be helpful for both: mentors and students
during Google Summer of Code. We have created an article on this topic
(i.e. Flower Dev Center + GSoC): [2], [3].

If you have a couple of spare minutes, could you tell us if you would
like to use Flower Dev Center? And/or raise topics that you think are
important (based on previous GSoC participations) to be supported by
Flower Dev Center?

The next version of Flower Dev Center, the 2.0.0 planned for June/July
2013, has major new features, that we did not demonstrate yet and that
will improve even more the collaboration between developers.

Flower Dev Center doesn't currently support Python programming language.
But if you are interested in using Flower Dev Center, we'll prioritize
it's implementation and we'll work hard to do it ASAP, with Flower Dev
Center 2.0.0.

Thank you in advance!

Best regards,
Mircea < at > The Flower Platform Team.

[1] - Flower Dev Center web site - http://www.flower-platform.com
[2] - Video “Flower Dev Center + Google Summer of Code” -
http://learn-discuss.flower-platform.com/flower_dev_center/videos/flower_dev_center_and_google_summer_of_code_2013_the_video
[3] - Text version with same content as the above video -
http://learn-discuss.flower-platform.com/flower_dev_center/tutorials/flower_dev_center_and_google_summer_of_code_2013

------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d

Re: How many project per master?

Benoît Allard — 2013-05-15T10:03:56



It's just a matter of specifying the right slave name in the builder settings under the 'slavenames' key [1], and then the builder is linked to the slave. No matter what other builder are already linked to that slave.

[1] http://docs.buildbot.net/latest/developer/config.html#buildbot.config.BuilderConfig.slavenames

Regards,
Benoît.
------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d_______________________________________________
Buildbot-devel mailing list
Buildbot-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
https://lists.sourceforge.net/lists/listinfo/buildbot-devel

Template folder

Simon Holm Nielsen — 2013-05-15T09:38:06

Hello

Maybe i should explain in details what I want to achieve.

Im working on the UI of a buildbot installation. My html files are located in the status/web/templates folder.

I have downloaded buildbot version Nine to get some inspiration and can see that the UI has another location. buildbot / www /  src  

I want to keep the default html files in status/web/templates folder unchanged and make a new UI in another folder (running Buildbot version: 0.8.7p1).. 

How do I do that?

I already tried overwriting the default templates by copying them to the files folder but that did not work.

Greetings,

Simon 

------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d_______________________________________________
Buildbot-devel mailing list
Buildbot-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
https://lists.sourceforge.net/lists/listinfo/buildbot-devel

Buildbot nine template path

Simon Holm Nielsen — 2013-05-15T08:48:49

Hello

How and where do I change the url to the templatefolder?  In buildbot Nine The templates are located in buildbot / www /  src  

in stead of the default status/web/templates folder


Greetings,

Simon ------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d_______________________________________________
Buildbot-devel mailing list
Buildbot-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
https://lists.sourceforge.net/lists/listinfo/buildbot-devel

Grace Hopper Open Source Day

Dustin J. Mitchell — 2013-05-15T02:36:45

Lukas Blakk contacted me to invite Buildbot to the 2013 Grace Hopper
Open Source Day in Minneapolis, MN on October 5.  The event would be
similar to the Buildbot sprints at PyCon, but shorter (one day) and
focused on new contributors.  In particular, this event would provide
an opportunity for women to hack on Buildbot and connect with the
Buildbot community.

This will be part of the Grace Hopper Celebration of Women in
Computing - http://gracehopper.org/2013/.  The info for this year's
GHOSD isn't up yet, but last year's is described here:
  http://gracehopper.org/2012/conference/grace-hopper-open-source-day/

I'd love to see Buildbot represented here, but personal complications
make it hard to know if I'll be able to attend.  If you can be there,
please let me know so that I can let the organizers know that Buildbot
will be participating.

Dustin

------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d

Re: How many project per master?

Diane Maple — 2013-05-14T22:38:51

inline, prefix with '--'

Thank you

________________________________
 From: Marc-Antoine Ruel <maruel-F7+t8E8rja9g9hUCZPvPmw< at >public.gmane.org>
To: Diane Maple <dianemaple76-/E1597aS9LQAvxtiuMwx3w< at >public.gmane.org> 
Cc: "Buildbot-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org" <Buildbot-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org> 
Sent: Tuesday, May 14, 2013 3:22 PM
Subject: Re: [Buildbot-devel] How many project per master?

As a random example, the chromium try server, visible at http://build.chromium.org/p/tryserver.chromium/waterfall, has around 700 slaves connected and it's baring the load with pain. Note that it's running on a fairly powerful machine with 64gb of ram but increasing the number of cpu cores won't help since buildbot is mostly single-threaded.

Re: How many project per master?

Marc-Antoine Ruel — 2013-05-14T22:22:32

As a random example, the chromium try server, visible at
http://build.chromium.org/p/tryserver.chromium/waterfall, has around 700
slaves connected and it's baring the load with pain. Note that it's running
on a fairly powerful machine with 64gb of ram but increasing the number of
cpu cores won't help since buildbot is mostly single-threaded.

Note that you may hit another issue, the load on the subversion server
itself, depending on your use case, e.g. how many gb each checkout
represents. You may want to create a read-only svnsync'ed clone if the
server is not powerful enough.

So overall, it should be fine.

We never run multiple buildslave instance on a single VM, we use slave
sharing instead; e.g. a single buildslave is connected to multiple builders.

M-A


2013/5/14 Diane Maple <dianemaple76-/E1597aS9LQAvxtiuMwx3w< at >public.gmane.org>

------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d_______________________________________________
Buildbot-devel mailing list
Buildbot-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
https://lists.sourceforge.net/lists/listinfo/buildbot-devel

How many project per master?

Diane Maple — 2013-05-14T21:54:37


Hello,

How many projects can I have controlled by one master?

I currently have 5 masters for different projects.
Each master has between 26 and 48 builds to control.
I was thinking of merging all of them into one.
I just want to make sure buildbot will be okay with this.

My goal is to have my 14 slave systems have 8 builder processes each.
These 8 builders building any of the projects that need to be built.

As of now I have configured the buildbot master to have two slaves for each master on each slave node.

The problem is the slaves are either under used or over used depending on what is happening in subversion.

Thank you for your comments.

Diane------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d_______________________________________________
Buildbot-devel mailing list
Buildbot-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f< at >public.gmane.org
https://lists.sourceforge.net/lists/listinfo/buildbot-devel