gmane.linux.kernel.tracing

Re: [lttng-dev] [PATCH] configure: exitifurcu/uatomic.h is not found

Paul E. McKenney — 2012-05-20T15:56:11

Another approach is to have sheepdog simply download the desired version
of liburcu if it doesn't like the one it finds.  ;-)

Thanx, Paul

Re: [lttng-dev] [PATCH] configure: exitifurcu/uatomic.h is not found

Mathieu Desnoyers — 2012-05-20T15:28:29

Hi Christoph,

Hrm, 0.4.1 is really old (2010-02-12). We did migrate some header names
and clean up some API namespacing back then between 0.4 and 0.5, so we
would not have to do it when the project would get more exposure. In
terms of way forward, we were planning to remove urcu/uatomic_arch.h
after a rather long deprecation period, being replaced by
urcu/uatomic.h. As you noticed, there has been a deprecation #warning in
place since the move has been done.

If you want to keep compatibility with the old 0.4.x version of liburcu
for a longer time, my recommendation would be to add a configure.ac
check for urcu/uatomic.h. It it is missing, then check for
urcu/uatomic_arch.h. This would specify a preprocessor macro that will
select either in a wrapper header within the sheepdog project.
If you only rely on uatomic, doing so should handle deprecation of the
older liburcu 0.4.x support within sheepdog at some point in the future
if need be.

It's good to be aware that between 0.4.x and 0.5, a namespace cleanup
for all the smp_*mb() went in: those are now all prefixed with "cmm_",
e.g.  cmm_smp_mb(). Same for a few other API members that were clashing
with private project namespaces (all documented in the ChangeLog).
uatomic_*() already had its namespace, so it did not need to be changed.

If in the future sheepdog starts using features for which the namespace
changed at 0.5, you might want to consider dropping support for liburcu
0.4.x.

Whether you decide on just bailing out is urcu/uatomic.h is not found or
use a detection macro is up to you, and depends on how far back you want
to support old distros.

Thanks!

Mathieu

[CFP] Tracing Micro-conference at LPC2012 (August 28-30, San Diego)

Mathieu Desnoyers — 2012-05-15T23:48:26

Hi,

We are organizing a tracing micro-conference taking place in San Diego,
within Linux Plumbers Conference 2012. The micro-conference is planned
to be held somewhere between August 28 and 30, in San Diego, CA.

If you are interested to present, please don't hesitate to get in touch
with us quickly (ideally within this week, or next week), since
deadlines are approaching.

Our intent is to gather people involved in development and users of
tracing tools and trace analysis tools to allow discussing the latest
developments, allow users to express their needs, and generally ensure
that developers and end users understand each other.

We are welcoming presentations from both end users and developers, on
topics covering, but not limited to:

- Trace collection and extraction,
- Trace filtering,
- Trace aggregation,
- Trace formats,
- Tracing multi-core systems,
- Trace abstraction,
- Trace modeling,
- Automated trace analysis (e.g. dependency analysis),
- Tracing large clusters and distributed systems,
- Hardware-level tracing (e.g. DSP, GPU, bare-metal),
- Trace visualisation,
- Interaction between debugging and tracing,
- Tracing remote control,
- Analysis of large trace datasets.

It can cover both recently available technologies and ongoing work.

If you are interested to present, please get in touch with Dominique
Toupin or myself. The micro-conference wiki can be accessed at:
http://wiki.linuxplumbersconf.org/2012:tracing

Feedback is welcome!

Thank you,

Dominique Toupin & Mathieu Desnoyers

[RELEASE] LTTng 2.0 prerelease bundle 20120202

Mathieu Desnoyers — 2012-02-03T03:43:46

LTTng, the Linux Trace Toolkit Next Generation, is a highly efficient
full system tracing solution toolchain. It is composed of several
components to allow tracing of the kernel, of userspace, trace viewing
and analysis and trace streaming. LTTng is open source software.

This bundle (20120202) contains:

lttng-modules-2.0-pre12.tar.bz2   - Linux kernel tracer
lttng-ust-1.9.5.tar.gz            - User-space tracing
lttng-tools-2.0-pre19.tar.bz2     - Tracer control (for kernel and
                                    userspace tracing)
babeltrace-0.9.tar.bz2            - Trace to text pretty printer
userspace-rcu-0.6.7.tar.bz2       - User-level synchronization lib

This will be the last prerelease bundle before Release Candidate 1. From
rc1, LTTng 2.0 will enter in feature-freeze mode until 2.0 final, which
is expected around Feb. 15. Testing is therefore more than very
welcome!

You can get a the current release "bundle" (recommanded set of packages)
at the following URL:

http://lttng.org/bundles/

Project website: http://lttng.org
Download link: http://lttng.org/lttng2.0
(please refer to the README files for installation instructions and
lttng-tools doc/quickstart.txt for usage information)

Re: [lttng-dev] Perf ABI (was: Re: [PATCH 09/11] sched: exporttask_prio to GPL modules)

Mathieu Desnoyers — 2012-01-12T16:27:56

OK. Then how can trace-cmd support the LTTng features ?

Thanks,

Mathieu

Re: Perf ABI (was: Re: [lttng-dev] [PATCH 09/11] sched: exporttask_prio to GPL modules)

Mathieu Desnoyers — 2012-01-12T14:09:06

[...]

Please allow me to look into what needs to be kept compatible for a good
user experience (for both Linux end users and kernel developers) in the
case of tracing:

Let's first describe what we really utterly don't want: random breakages
between the kernel and user-level tracing control/transport/analysis
tools. Consequently, I think we could say that it would be unacceptable
for userspace tools to break for every slight change of kernel code. If
that would be the case (as it was with the approach SystemTap was taking
before they started hooking into the kernel with tracepoints), then we'd
need to regenerate the tools for pretty much every -rc kernel, and for
each local development tree, which would make those tools useless to
kernel developers.

It is important to clarify that tracing is, in my opinion, not part of
the runtime support, which makes it very different by nature from
filesystems and kernel runtime support. So I agree with Linus' argument
about not breaking userspace when applied to runtime support, because
being unable to even boot a system due to an ABI breakage is very much
unwanted. However, I think it should not be applied as-is to tracing,
because you cannot make a system unusable due to a tracer ABI breakage:
if a tracer can be packaged in a set of standalone modules, that clearly
shows it is not part of the system runtime support.

That being said, ABI versioning could still handle ABI changes without
significantly impacting the users: when an ABI breakage is needed, we
can keep the old code around for a while and expose both the old and new
ABIs. This would ensure that the user-level tools can query for the
specific ABI major version(s) they support. That should improve the user
experience by providing "deprecated" console warnings for a few kernel
releases before the old code ends up being removed.

So, in summary:

  * Old kernels vs new tools:

New tools can query for the latest ABI they know, and fall-back on older
ABIs, with limited features.

  * New kernels vs old tools:

Keeping around the old ABI for a deprecation phase lets old tools work on
a bleeding edge kernel while the ABI change is being introduced, which
should satisfy the kernel developer use-case.

Best regards,

Mathieu

Re: [lttng-dev] Perf ABI (was: Re: [PATCH 09/11] sched: exporttask_prio to GPL modules)

Mathieu Desnoyers — 2012-01-12T16:27:56

OK. Then how can trace-cmd support the LTTng features ?

Thanks,

Mathieu

Re: Perf ABI (was: Re: [lttng-dev] [PATCH 09/11] sched: exporttask_prio to GPL modules)

Mathieu Desnoyers — 2012-01-12T14:09:06

[...]

Please allow me to look into what needs to be kept compatible for a good
user experience (for both Linux end users and kernel developers) in the
case of tracing:

Let's first describe what we really utterly don't want: random breakages
between the kernel and user-level tracing control/transport/analysis
tools. Consequently, I think we could say that it would be unacceptable
for userspace tools to break for every slight change of kernel code. If
that would be the case (as it was with the approach SystemTap was taking
before they started hooking into the kernel with tracepoints), then we'd
need to regenerate the tools for pretty much every -rc kernel, and for
each local development tree, which would make those tools useless to
kernel developers.

It is important to clarify that tracing is, in my opinion, not part of
the runtime support, which makes it very different by nature from
filesystems and kernel runtime support. So I agree with Linus' argument
about not breaking userspace when applied to runtime support, because
being unable to even boot a system due to an ABI breakage is very much
unwanted. However, I think it should not be applied as-is to tracing,
because you cannot make a system unusable due to a tracer ABI breakage:
if a tracer can be packaged in a set of standalone modules, that clearly
shows it is not part of the system runtime support.

That being said, ABI versioning could still handle ABI changes without
significantly impacting the users: when an ABI breakage is needed, we
can keep the old code around for a while and expose both the old and new
ABIs. This would ensure that the user-level tools can query for the
specific ABI major version(s) they support. That should improve the user
experience by providing "deprecated" console warnings for a few kernel
releases before the old code ends up being removed.

So, in summary:

  * Old kernels vs new tools:

New tools can query for the latest ABI they know, and fall-back on older
ABIs, with limited features.

  * New kernels vs old tools:

Keeping around the old ABI for a deprecation phase lets old tools work on
a bleeding edge kernel while the ABI change is being introduced, which
should satisfy the kernel developer use-case.

Best regards,

Mathieu

Re: [lttng-dev] [RFC PATCH v2 0/4] simpletrace : support var num of args and strings.

Stefan Hajnoczi — 2012-01-11T09:30:38

On Tue, Jan 10, 2012 at 5:29 PM, Mathieu Desnoyers
<mathieu.desnoyers< at >efficios.com> wrote:

I see, thanks for explaining.


SystemTap has a solution for this problem where you wrap the
tracepoint with a guard:

if (QEMU_FOO_ENABLED()) {
    /* ...expensive stuff here... */
    QEMU_FOO(a, b, c);
}

In QEMU we don't really make use of this but I've heard applications
with expensive tracepoint arguments use this.


Okay, QEMU can continue to emit LTTng UST tracepoints.  I thought you
were planning to make UST source-compatible with DTrace sdt.h, which
would be convenient for applications that are already DTrace
instrumented but want a fast shared-memory mechanism instead of
breakpoints - they could use ust-dtrace(1) instead of dtrace(1) and
magically work with UST.  In QEMU's case it doesn't matter because
we're already geared towards multiple tracing backends.

Stefan

Re: [lttng-dev] [RFC PATCH v2 0/4] simpletrace : support var num of args and strings.

Stefan Hajnoczi — 2012-01-10T14:58:02

On Tue, Jan 10, 2012 at 12:14 AM, Mathieu Desnoyers
<mathieu.desnoyers< at >efficios.com> wrote:

If lttng supports sdt.h in the future will it also provide a dtrace(1)
wrapper?  I'm wondering if we can boil it down to the common DTrace
code that we already use in QEMU for SystemTap and that the
SmartOS/Illumos folks have used with DTrace.

Stefan

Re: [lttng-dev] [RFC PATCH v2 0/4] simpletrace :support var numof args and strings.

Mathieu Desnoyers — 2012-01-10T17:29:05

Hi Stefan,

* Stefan Hajnoczi (stefanha< at >gmail.com) wrote:

What we plan to provide is the other way around: because we want to keep
the TRACEPOINT_EVENT description, which specify the mapping between the
arguments passed to the tracepoint() and what is to be serialized into
the trace, as part of the application code, we require that the
instrumentation be specified in the form of TRACEPOINT_EVENT and called
with tracepoint(). The bonus here is that the tracepoint() macro can
embed a STAP_PROBEV() call, which should be compatible with systemtap
and gdb.

Another point worth noting is that sdt provider uses a breakpoint to
call the probes from the application instrumentation sites, which brings
an added overhead compared to tracepoints when it is activated. This is
another reason for staying with tracepoints and not use SDT as is within
lttng-ust.

One more point is that tracepoints put all side-effects of the
parameters passed to the tracepoint() macro inside the "instrumentation
enabled" code path (pointer dereferenced, offsets computation, functions
called...). Therefore, when tracepoints are disabled, they just cost a
branch (which we plan to optimize to a no-op, as permitted by gcc asm
goto).

In theory, it might not be impossible to create a translator from a
subset of the dtrace language to TRACEPOINT_EVENT declarations. It is
just not on our roadmap at this point, and I'm not sure it's worth the
effort, when a simple script could translate from qemu-kvm trace-event
file to TRACEPOINT_EVENT declarations.

Thoughts ?

Best regards,

Mathieu

Re: [RFC PATCH v2 0/4] simpletrace : support var num of args and strings.

Harsh Bora — 2012-01-10T06:54:09

I had tried with _NOARGS variants initially by manually changing the 
auto-generated code like this:

In trace.h:

DECLARE_TRACE_NOARGS(ust_slavio_misc_update_irq_raise);
#define trace_slavio_misc_update_irq_raise 
trace_ust_slavio_misc_update_irq_raise

In trace.c:

DEFINE_TRACE_NOARGS(ust_slavio_misc_update_irq_raise);

static void ust_slavio_misc_update_irq_raise_probe()
{
     trace_mark(ust, slavio_misc_update_irq_raise);
}


However, it still gave error like this:

[harsh< at >harshbora v9fs]$ make
   CC    osdep.o
cc1: warnings being treated as errors
In file included from osdep.c:49:
trace.h:277: error: data definition has no type or storage class
trace.h:277: error: type defaults to ‘int’ in declaration of 
‘DECLARE_TRACE_NOARGS’
trace.h:277: error: parameter names (without types) in function declaration


It will be great if you can provide a sample code (to be auto-generated) 
required for a trace-event with void argument.

- Harsh


Thanks for the info, I will look into it later.

Re: [RFC PATCH v2 0/4] simpletrace : support var numof args andstrings.

Mathieu Desnoyers — 2012-01-09T16:01:05

Hi,

FYI, the LTTng-UST TRACEPOINT_EVENT API is very much stable as of now.
Even though we are still in LTTng-UST 2.0 prereleases, the fact that we
started the round of discussions on this API last summer makes us
confident that from this point on we should not have to change it.

Moreover, I would like to know if the old UST 0.x (0.16 is the latest)
is broken wrt qemu, or if this is just for LTTng-2.0 UST support ?
UST 0.x instrumentation is not supposed to have broken wrt qemu.

Best regards,

Mathieu

Re: [RFC PATCH v2 0/4] simpletrace : support var num of args and strings.

Harsh Bora — 2012-01-10T06:54:09

I had tried with _NOARGS variants initially by manually changing the 
auto-generated code like this:

In trace.h:

DECLARE_TRACE_NOARGS(ust_slavio_misc_update_irq_raise);
#define trace_slavio_misc_update_irq_raise 
trace_ust_slavio_misc_update_irq_raise

In trace.c:

DEFINE_TRACE_NOARGS(ust_slavio_misc_update_irq_raise);

static void ust_slavio_misc_update_irq_raise_probe()
{
     trace_mark(ust, slavio_misc_update_irq_raise);
}


However, it still gave error like this:

[harsh< at >harshbora v9fs]$ make
   CC    osdep.o
cc1: warnings being treated as errors
In file included from osdep.c:49:
trace.h:277: error: data definition has no type or storage class
trace.h:277: error: type defaults to ‘int’ in declaration of 
‘DECLARE_TRACE_NOARGS’
trace.h:277: error: parameter names (without types) in function declaration


It will be great if you can provide a sample code (to be auto-generated) 
required for a trace-event with void argument.

- Harsh


Thanks for the info, I will look into it later.

Re: [RFC PATCH v2 0/4] simpletrace : support var numof args andstrings.

Mathieu Desnoyers — 2012-01-09T16:01:05

Hi,

FYI, the LTTng-UST TRACEPOINT_EVENT API is very much stable as of now.
Even though we are still in LTTng-UST 2.0 prereleases, the fact that we
started the round of discussions on this API last summer makes us
confident that from this point on we should not have to change it.

Moreover, I would like to know if the old UST 0.x (0.16 is the latest)
is broken wrt qemu, or if this is just for LTTng-2.0 UST support ?
UST 0.x instrumentation is not supposed to have broken wrt qemu.

Best regards,

Mathieu

Re: Perf ABI (was: Re: [lttng-dev] [PATCH 09/11] sched: exporttask_prio to GPL modules)

Mathieu Desnoyers — 2011-12-23T18:16:41

Hi Ted,

* Ted Ts'o (tytso< at >mit.edu) wrote:

I completely agree that systemtap did not have the right level of
compatibility towards changes. It clearly does not make sense to require
the tools to be updated whenever the kernel version and instrumentation
changes. What makes sense to me, though, is to allow breakage when a
newly introduced tracer feature requires the ABI to break.

What I currently see as a tradeoff sweet-spot between compatibility
burden and ability to innovate is to split the ABI and handle
compatibility as follows:

- ABIs to control the tracer
  - Versioned, ideally always incrementally adding features, but still
    keeping room for major changes if needed. We should expect very,
    very seldom breakages on this front. This requires update of tracer
    control tools when the ABI is broken.

- ABIs to transport tracing data
  - Versioned, can and should change when a feature or transport
    performance enhancement require to break compatibility. This
    requires update of trace data consumer tools when compability is
    broken.

(note that ABI to control the tracer and ABI to transport data could
share the same version numbering if the control tools and transport
tools happen to reside in the same user-level packages)

- The trace data format
  - Both versioned _and_ self-described.
  Self-description of the event/field layout allows the same tools to
  understand traces gathered on different kernel versions, on different
  architectures, with different tracer configurations.
  Versioning on top of the self-described trace format allows changes
  to what the trace self-description can express.

So the breakages would happen only when required by tracer tool
capability enhancements, not randomly when a kernel instrumentation
source happens to change.

Best regards,

Mathieu

P.S.: my next replies will be slightly delayed, due to Christmas
holidays.

Re: Perf ABI (was: Re: [lttng-dev] [PATCH 09/11] sched: exporttask_prio to GPL modules)

Mathieu Desnoyers — 2011-12-23T18:16:41

Hi Ted,

* Ted Ts'o (tytso< at >mit.edu) wrote:

I completely agree that systemtap did not have the right level of
compatibility towards changes. It clearly does not make sense to require
the tools to be updated whenever the kernel version and instrumentation
changes. What makes sense to me, though, is to allow breakage when a
newly introduced tracer feature requires the ABI to break.

What I currently see as a tradeoff sweet-spot between compatibility
burden and ability to innovate is to split the ABI and handle
compatibility as follows:

- ABIs to control the tracer
  - Versioned, ideally always incrementally adding features, but still
    keeping room for major changes if needed. We should expect very,
    very seldom breakages on this front. This requires update of tracer
    control tools when the ABI is broken.

- ABIs to transport tracing data
  - Versioned, can and should change when a feature or transport
    performance enhancement require to break compatibility. This
    requires update of trace data consumer tools when compability is
    broken.

(note that ABI to control the tracer and ABI to transport data could
share the same version numbering if the control tools and transport
tools happen to reside in the same user-level packages)

- The trace data format
  - Both versioned _and_ self-described.
  Self-description of the event/field layout allows the same tools to
  understand traces gathered on different kernel versions, on different
  architectures, with different tracer configurations.
  Versioning on top of the self-described trace format allows changes
  to what the trace self-description can express.

So the breakages would happen only when required by tracer tool
capability enhancements, not randomly when a kernel instrumentation
source happens to change.

Best regards,

Mathieu

P.S.: my next replies will be slightly delayed, due to Christmas
holidays.

Re: [lttng-dev] [PATCH 09/11] sched: export task_prio to GPL modules

Ingo Molnar — 2011-12-23T10:51:28

* Frank Rowand <frank.rowand< at >am.sony.com> wrote:


I'm not saying that it's absolutely never done: for example 
monitoring/logging on a production box and evaluating events 
only once per month would certainly qualify.

I just say that the overwhelming majority of usecases utilize 
traces on a short time-span and that we must keep the common 
usecase in mind when supporting not so common usecases.

It's the same deal as with -rt: compared to the 'normal' usage 
of Linux -rt is somewhat of a special case - yet it's still 
something very much worth doing, as long as the main usecase is 
always kept in mind.

Thanks,

Ingo

Re: [lttng-dev] [PATCH 09/11] sched: export task_prio to GPL modules

Ingo Molnar — 2011-12-23T10:51:28

* Frank Rowand <frank.rowand< at >am.sony.com> wrote:


I'm not saying that it's absolutely never done: for example 
monitoring/logging on a production box and evaluating events 
only once per month would certainly qualify.

I just say that the overwhelming majority of usecases utilize 
traces on a short time-span and that we must keep the common 
usecase in mind when supporting not so common usecases.

It's the same deal as with -rt: compared to the 'normal' usage 
of Linux -rt is somewhat of a special case - yet it's still 
something very much worth doing, as long as the main usecase is 
always kept in mind.

Thanks,

Ingo

[PATCH] staging: Remove LTTng from MAINTAINERS file

Mathieu Desnoyers — 2011-12-21T21:48:55

LTTng has been removed from the staging tree. Complete this removal by
removing the LTTng entry from the MAINTAINERS file.
    
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers< at >efficios.com>
CC: Greg Kroah-Hartman <gregkh< at >suse.de>
CC: Ingo Molnar <mingo< at >elte.hu>
CC: Peter Zijlstra <peterz< at >infradead.org>
---
diff --git a/MAINTAINERS b/MAINTAINERS
index f3bb825..1514f7a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
< at >< at > -4132,13 +4132,6 < at >< at > W:http://ltp.sourceforge.net/
 T:git git://ltp.git.sourceforge.net/gitroot/ltp/ltp-dev
 S:Maintained
 
-LTTng (Linux Trace Toolkit Next Generation)
-M:Mathieu Desnoyers <mathieu.desnoyers< at >efficios.com>
-L:lttng-dev< at >lists.lttng.org (moderated for non-subscribers)
-W:http://lttng.org
-S:Maintained
-F:drivers/staging/lttng/
-
 M32R ARCHITECTURE
 M:Hirokazu Takata <takata< at >linux-m32r.org>
 L:linux-m32r< at >ml.linux-m32r.org (moderated for non-subscribers)

[PATCH] staging: Remove LTTng from MAINTAINERS file

Mathieu Desnoyers — 2011-12-21T21:48:55

LTTng has been removed from the staging tree. Complete this removal by
removing the LTTng entry from the MAINTAINERS file.
    
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers< at >efficios.com>
CC: Greg Kroah-Hartman <gregkh< at >suse.de>
CC: Ingo Molnar <mingo< at >elte.hu>
CC: Peter Zijlstra <peterz< at >infradead.org>
---
diff --git a/MAINTAINERS b/MAINTAINERS
index f3bb825..1514f7a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
< at >< at > -4132,13 +4132,6 < at >< at > W:http://ltp.sourceforge.net/
 T:git git://ltp.git.sourceforge.net/gitroot/ltp/ltp-dev
 S:Maintained
 
-LTTng (Linux Trace Toolkit Next Generation)
-M:Mathieu Desnoyers <mathieu.desnoyers< at >efficios.com>
-L:lttng-dev< at >lists.lttng.org (moderated for non-subscribers)
-W:http://lttng.org
-S:Maintained
-F:drivers/staging/lttng/
-
 M32R ARCHITECTURE
 M:Hirokazu Takata <takata< at >linux-m32r.org>
 L:linux-m32r< at >ml.linux-m32r.org (moderated for non-subscribers)