gmane.linux.oprofile
http://blog.gmane.org/gmane.linux.oprofile
hourly11901-01-01T00:00+00:00Gmanehttp://gmane.org/img/gmane-25t.png
http://gmane.org
[PATCH] Fix up anonymous mapping support in operf to include vdso,stack, and heap
http://comments.gmane.org/gmane.linux.oprofile/10509
<pre>[PATCH] Fix up anonymous mapping support in operf to include vdso, stack, and heap
Signed-off-by: Maynard Johnson <maynardj< at >us.ibm.com>
This patch has been pushed to the perf-events branch, but review comments are
welcome.
---
libperf_events/operf_mangling.cpp | 2 +-
libperf_events/operf_utils.cpp | 17 +++++++++++------
2 files changed, 12 insertions(+), 7 deletions(-)
diff --git a/libperf_events/operf_mangling.cpp b/libperf_events/operf_mangling.cpp
index 5b31e29..5165807 100644
--- a/libperf_events/operf_mangling.cpp
+++ b/libperf_events/operf_mangling.cpp
< at >< at > -63,7 +63,7 < at >< at > mangle_filename(struct operf_sfile * last, struct operf_sfile const * sf, int co
} else if (sf->is_anon) {
values.flags |= MANGLE_ANON;
values.image_name = mangle_anon(sf);
-values.anon_name = "anon";
+values.anon_name = sf->image_name;
} else {
values.image_name = sf->image_name;
}
diff --git a/libperf_events/operf_utils.cpp b/libperf_events/operf_utils.cpp
index 42eecce..9ee9b80 100644
--- a/libperf_events/operf_utils.cpp
+++ b/libperf_events/operf_utils.cpp
< at >< at > -208,12 +208,17 < at >< at > static void __handle_mmap_event(event_t * event)
memset(mapping, 0, sizeof(struct operf_mmap));
mapping->start_addr = event->mmap.start;
strcpy(mapping->filename, event->mmap.filename);
- mapping->is_anon_mapping = (strncmp(mapping->filename,
- "//anon",
- strlen("//anon")) == 0) ?
- true
- : false;
-
+/* Mappings starting with "/" are for either a file or shared memory object.
+ * From the kernel's perf_events subsystem, anon maps have labels like:
+ * [heap], [stack], [vdso], //anon
+ */
+if (mapping->filename[0] == '[') {
+mapping->is_anon_mapping = true;
+} else if ((strncmp(mapping->filename, "//anon",
+ strlen("//anon")) == 0)) {
+mapping->is_anon_mapping = true;
+strcpy(mapping->filename, "anon");
+}
mapping->end_addr = (event->mmap.len == 0ULL)? 0ULL : mapping->start_addr + event->mmap.len - 1;
mapping->pgoff = event->mmap.pgoff;
</pre>Maynard Johnson2012-05-25T14:00:32[ oprofile-Bugs-3529111 ] opjitconv failed to parse JIT dump header
http://comments.gmane.org/gmane.linux.oprofile/10507
<pre>Bugs item #3529111, was opened at 2012-05-23 08:05
Message generated for change (Tracker Item Submitted) made by
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=116191&aid=3529111&group_id=16191
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: https://www.google.com/accounts ()
Assigned to: Nobody/Anonymous (nobody)
Summary: opjitconv failed to parse JIT dump header
Initial Comment:
The bug occurs on x86_64 Linux (Fedora 15) with oprofile installed from oprofile-0.9.6-21.fc15.x86_64.rpm while trying to profile JIT code produced by 32-bit application.
The debug output of opjitconv shows:
header: bfd-arch=9, bfd-mach=1, bfd_target_name=2-i386
Note that target name must be "elf32-i386"
I assume that the "u64 timestamp" field of "jitheader" structure is 8-byte aligned and as result field "bfd_target" is shifted 4 bytes forward.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=116191&aid=3529111&group_id=16191
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
</pre>SourceForge.net2012-05-23T15:05:08[PATCH 1/2] Implement the ANY Intel extra bit
http://comments.gmane.org/gmane.linux.oprofile/10506
<pre>From: Andi Kleen <ak< at >linux.intel.com>
Implement the ANY (any thread) extra bit for Intel CPUs. Needed
for some of the new events.
Signed-off-by: Andi Kleen <ak< at >linux.intel.com>
---
libop/op_events.c | 3 +++
libop/op_events.h | 1 +
2 files changed, 4 insertions(+), 0 deletions(-)
diff --git a/libop/op_events.c b/libop/op_events.c
index bbb0130..49b9563 100644
--- a/libop/op_events.c
+++ b/libop/op_events.c
< at >< at > -127,6 +127,9 < at >< at > unsigned parse_extra(const char *s)
} else if (sscanf(s, "cmask=%x%n", &w, &o) >= 1) {
v |= (w & EXTRA_CMASK_MASK) << EXTRA_CMASK_SHIFT;
s += o;
+} else if (strisprefix(s, "any")) {
+v |= EXTRA_ANY;
+s += 3;
} else {
parse_error("Illegal extra field modifier");
}
diff --git a/libop/op_events.h b/libop/op_events.h
index 8cbb622..1a65a7a 100644
--- a/libop/op_events.h
+++ b/libop/op_events.h
< at >< at > -21,6 +21,7 < at >< at > extern "C" {
#include "op_list.h"
#define EXTRA_EDGE (1U << 18)
+#define EXTRA_ANY (1U << 21)
#define EXTRA_INV (1U << 23)
#define EXTRA_CMASK_SHIFT 24
#define EXTRA_CMASK_MASK 0xff
</pre>Andi Kleen2012-05-23T00:48:42[PATCH] Change operf to create oprofile sample files during profilinginstead of afterwards
http://comments.gmane.org/gmane.linux.oprofile/10504
<pre>[PATCH] Change operf to create oprofile sample files during profiling instead of afterwards
Previously, the 'operf-record' process read the perf_events sample data from the kernel
and wrote it to a temporary output file (operf.data). Then, once the profiling session
was completed, a conversion was performed on that data to put it into oprofile format,
stored in the usual sample files in <session-dir>/samples/current. I noticed that when
operf was run in system-wide mode, the temporary operf.data file was very large, and the
conversion step was taking inordinately long (from several seconds on a laptop to over
a minute on a large, busy server system.
I've had a TODO comment in the operf.cpp code to investigate performing the conversion
step in-flight (while profiling) versus waiting for the profiling to end. So I changed
the operf-record process from writing the perf_events sample data to a file to writing
to a pipe. Then I created a new operf-read process to read from the other end of that
pipe and perform the conversion to oprofile format. Doing the conversion during the
profiling run allowed me to identify the hot spots in the opreport output. I found several
areas (especially involving the inefficiency of the operf_sfile hash algorithm) where
improvements were needed. The conversion step now takes about 15% of the time it used
to! A wonderful result!
This patch has been committed to the perf-events branch, but review comments are welcome.
Signed-off-by: Maynard Johnson <maynardj< at >us.ibm.com>
---
libperf_events/operf_counter.cpp | 214 ++++++++++++++++++----------------
libperf_events/operf_counter.h | 12 +-
libperf_events/operf_event.h | 11 --
libperf_events/operf_sfile.cpp | 74 ++++++------
libperf_events/operf_sfile.h | 2 +
libperf_events/operf_utils.cpp | 237 +++++++++++++-------------------------
libperf_events/operf_utils.h | 12 +--
pe_profiling/operf.cpp | 168 ++++++++++++++++++++-------
8 files changed, 366 insertions(+), 364 deletions(-)
diff --git a/libperf_events/operf_counter.cpp b/libperf_events/operf_counter.cpp
index 1ceb012..5d1f016 100644
--- a/libperf_events/operf_counter.cpp
+++ b/libperf_events/operf_counter.cpp
< at >< at > -36,6 +36,7 < at >< at > using namespace OP_perf_utils;
volatile bool quit;
+volatile bool read_quit;
int sample_reads;
unsigned int pagesize;
verbose vperf("perf_events");
< at >< at > -121,17 +122,14 < at >< at > int operf_counter::perf_event_open(pid_t ppid, int cpu, unsigned event, operf_re
operf_record::~operf_record()
{
cverb << vperf << "operf_record::~operf_record()" << endl;
-opHeader.data_size = total_bytes_recorded;
-if (total_bytes_recorded)
-write_op_header_info();
delete[] poll_data;
-close(outputFile);
+close(output_fd);
samples_array.clear();
evts.clear();
perfCounters.clear();
}
-operf_record::operf_record(string outfile, bool sys_wide, pid_t the_pid, bool pid_running,
+operf_record::operf_record(int out_fd, bool sys_wide, pid_t the_pid, bool pid_running,
vector<operf_event_t> & events, vmlinux_info_t vi, bool do_cg,
bool separate_by_cpu)
{
< at >< at > -154,14 +152,8 < at >< at > operf_record::operf_record(string outfile, bool sys_wide, pid_t the_pid, bool pi
if (system_wide && (pid != -1 || pid_started))
return; // object is not valid
-opHeader.data_size = 0;
-outputFile = open(outfile.c_str(), flags, S_IRUSR|S_IWUSR);
-if (outputFile < 0) {
-string errmsg = "Internal error: Could not create output file. errno is ";
-errmsg += strerror(errno);
-throw runtime_error(errmsg);
-}
-cverb << vperf << "operf_record ctor: successfully opened output file " << outfile << endl;
+output_fd = out_fd;
+cverb << vperf << "operf_record ctor using output fd " << output_fd << endl;
memset(&sa, 0, sizeof(struct sigaction));
sa.sa_sigaction = op_perfrecord_sigusr1_handler;
< at >< at > -198,37 +190,25 < at >< at > void operf_record::write_op_header_info()
struct OP_file_header f_header;
struct op_file_attr f_attr;
-lseek(outputFile, sizeof(f_header), SEEK_SET);
-
-for (unsigned i = 0; i < evts.size(); i++) {
-opHeader.h_attrs[i].id_offset = lseek(outputFile, 0, SEEK_CUR);
-add_to_total(op_write_output(outputFile, &opHeader.h_attrs[i].ids[0],
-opHeader.h_attrs[i].ids.size() * sizeof(u64)));
-}
+f_header.magic = OP_MAGIC;
+f_header.size = sizeof(f_header);
+f_header.attr_size = sizeof(f_attr);
+f_header.attrs.size = evts.size() * sizeof(f_attr);
+f_header.data.size = 0;
-opHeader.attr_offset = lseek(outputFile, 0, SEEK_CUR);
+add_to_total(op_write_output(output_fd, &f_header, sizeof(f_header)));
for (unsigned i = 0; i < evts.size(); i++) {
struct op_header_evt_info attr = opHeader.h_attrs[i];
f_attr.attr = attr.attr;
-f_attr.ids.offset = attr.id_offset;
-f_attr.ids.size =attr.ids.size() * sizeof(u64);
-add_to_total(op_write_output(outputFile, &f_attr, sizeof(f_attr)));
+f_attr.ids.size = attr.ids.size() * sizeof(u64);
+add_to_total(op_write_output(output_fd, &f_attr, sizeof(f_attr)));
}
-opHeader.data_offset = lseek(outputFile, 0, SEEK_CUR);
-
-f_header.magic = OP_MAGIC;
-f_header.size = sizeof(f_header);
-f_header.attr_size = sizeof(f_attr);
-f_header.attrs.offset = opHeader.attr_offset;
-f_header.attrs.size = evts.size() * sizeof(f_attr);
-f_header.data.offset = opHeader.data_offset;
-f_header.data.size = opHeader.data_size;
-
-lseek(outputFile, 0, SEEK_SET);
-add_to_total(op_write_output(outputFile, &f_header, sizeof(f_header)));
-lseek(outputFile, opHeader.data_offset + opHeader.data_size, SEEK_SET);
+for (unsigned i = 0; i < evts.size(); i++) {
+add_to_total(op_write_output(output_fd, &opHeader.h_attrs[i].ids[0],
+ opHeader.h_attrs[i].ids.size() * sizeof(u64)));
+}
}
int operf_record::prepareToRecord(int counter, int cpu, int fd)
< at >< at > -360,7 +340,7 < at >< at > void operf_record::setup()
closedir(dir);
write_op_header_info();
if (pid_started || system_wide) {
-if (op_record_process_info(system_wide, pid, this, outputFile) < 0) {
+if (op_record_process_info(system_wide, pid, this, output_fd) < 0) {
for (int i = 0; i < num_cpus; i++) {
for (unsigned int evt = 0; evt < evts.size(); evt++)
ioctl(perfCounters[i][evt].get_fd(), PERF_EVENT_IOC_DISABLE);
< at >< at > -368,7 +348,7 < at >< at > void operf_record::setup()
goto error;
}
}
-op_record_kernel_info(vmlinux_file, kernel_start, kernel_end, outputFile, this);
+op_record_kernel_info(vmlinux_file, kernel_start, kernel_end, output_fd, this);
// Set bit to indicate we're set to go.
valid = true;
< at >< at > -376,7 +356,7 < at >< at > void operf_record::setup()
error:
delete[] poll_data;
-close(outputFile);
+close(output_fd);
if (rc != OP_PERF_HANDLED_ERROR)
throw runtime_error(err_msg);
}
< at >< at > -409,15 +389,33 < at >< at > void operf_record::recordPerfData(void)
cverb << vperf << "operf_record::recordPerfData received signal to quit." << endl;
}
}
+close(output_fd);
cverb << vdebug << "operf recording finished." << endl;
}
-void operf_read::init(string infilename, string samples_loc, op_cpu cputype, vector<operf_event_t> & events)
+void operf_read::init(int sample_data_pipe_fd, string samples_loc, op_cpu cputype, vector<operf_event_t> & events)
{
-inputFname = infilename;
+struct sigaction sa;
+sigset_t ss;
+sample_data_fd = sample_data_pipe_fd;
sampledir = samples_loc;
evts = events;
cpu_type = cputype;
+memset(&sa, 0, sizeof(struct sigaction));
+sa.sa_sigaction = op_perfread_sigusr1_handler;
+sigemptyset(&sa.sa_mask);
+sigemptyset(&ss);
+sigaddset(&ss, SIGUSR1);
+sigprocmask(SIG_UNBLOCK, &ss, NULL);
+sa.sa_mask = ss;
+sa.sa_flags = SA_NOCLDSTOP | SA_SIGINFO;
+cverb << vperf << "operf-read calling sigaction" << endl;
+if (sigaction(SIGUSR1, &sa, NULL) == -1) {
+cverb << vperf << "operf-read init: sigaction failed; errno is: "
+ << strerror(errno) << endl;
+_exit(EXIT_FAILURE);
+}
+
}
operf_read::~operf_read()
< at >< at > -427,72 +425,66 < at >< at > operf_read::~operf_read()
int operf_read::readPerfHeader(void)
{
-int ret = 0;
-
-opHeader.data_size = 0;
-istrm.open(inputFname.c_str(), ios_base::in);
-if (!istrm.good()) {
-return -1;
-}
-istrm.peek();
-if (istrm.eof()) {
-cverb << vperf << "operf_read::readPerfHeader: Empty profile data file." << endl;
-valid = false;
-return OP_PERF_HANDLED_ERROR;
-}
-cverb << vperf << "operf_read: successfully opened input file " << inputFname << endl;
-read_op_header_info_with_ifstream();
-valid = true;
-cverb << vperf << "Successfully read perf header" << endl;
-
-return ret;
-}
-
-void operf_read::read_op_header_info_with_ifstream(void)
-{
struct OP_file_header fheader;
-istrm.seekg(0, ios_base::beg);
-
-if (op_read_from_stream(istrm, (char *)&fheader, sizeof(fheader)) != sizeof(fheader)) {
-throw runtime_error("Error: input file " + inputFname + " does not have enough data for header");
+string errmsg;
+int num_fattrs;
+size_t fattr_size;
+vector<struct op_file_attr> f_attr_cache;
+
+errno = 0;
+if (read(sample_data_fd, &fheader, sizeof(fheader)) != sizeof(fheader)) {
+errmsg = "Error reading header on sample data pipe: " + string(strerror(errno));
+goto fail;
}
-if (memcmp(&fheader.magic, __op_magic, sizeof(fheader.magic)))
-throw runtime_error("Error: input file " + inputFname + " does not have expected header data");
+if (memcmp(&fheader.magic, __op_magic, sizeof(fheader.magic))) {
+errmsg = "Error: operf sample data does not have expected header data";
+goto fail;
+}
cverb << vperf << "operf magic number " << (char *)&fheader.magic << " matches expected __op_magic " << __op_magic << endl;
-opHeader.attr_offset = fheader.attrs.offset;
-opHeader.data_offset = fheader.data.offset;
-opHeader.data_size = fheader.data.size;
-size_t fattr_size = sizeof(struct op_file_attr);
+fattr_size = sizeof(struct op_file_attr);
if (fattr_size != fheader.attr_size) {
-string msg = "Error: perf_events binary incompatibility. Event data collection was apparently "
+errmsg = "Error: perf_events binary incompatibility. Event data collection was apparently "
"performed under a different kernel version than current.";
-throw runtime_error(msg);
+goto fail;
}
-int num_fattrs = fheader.attrs.size/fheader.attr_size;
+num_fattrs = fheader.attrs.size/fheader.attr_size;
cverb << vperf << "num_fattrs is " << num_fattrs << endl;
-istrm.seekg(opHeader.attr_offset, ios_base::beg);
for (int i = 0; i < num_fattrs; i++) {
struct op_file_attr f_attr;
streamsize fattr_size = sizeof(f_attr);
-if (op_read_from_stream(istrm, (char *)&f_attr, fattr_size) != fattr_size)
-throw runtime_error("Error: Unexpected end of input file " + inputFname + ".");
+if (read(sample_data_fd, (char *)&f_attr, fattr_size) != fattr_size) {
+errmsg = "Error reading file attr on sample data pipe: " + string(strerror(errno));
+goto fail;
+}
opHeader.h_attrs[i].attr = f_attr.attr;
-streampos next_f_attr = istrm.tellg();
+f_attr_cache.push_back(f_attr);
+}
+for (int i = 0; i < num_fattrs; i++) {
+vector<struct op_file_attr>::iterator it = f_attr_cache.begin();
+struct op_file_attr f_attr = *(it);
int num_ids = f_attr.ids.size/sizeof(u64);
-istrm.seekg(f_attr.ids.offset, ios_base::beg);
+
for (int id = 0; id < num_ids; id++) {
u64 perf_id;
streamsize perfid_size = sizeof(perf_id);
-if (op_read_from_stream(istrm, (char *)& perf_id, perfid_size) != perfid_size)
-throw runtime_error("Error: Unexpected end of input file " + inputFname + ".");
+if (read(sample_data_fd, (char *)& perf_id, perfid_size) != perfid_size) {
+errmsg = "Error reading perf ID on sample data pipe: " + string(strerror(errno));
+goto fail;
+}
cverb << vperf << "Perf header: id = " << hex << (unsigned long long)perf_id << endl;
opHeader.h_attrs[i].ids.push_back(perf_id);
}
-istrm.seekg(next_f_attr, ios_base::beg);
+
}
-istrm.close();
+valid = true;
+cverb << vperf << "Successfully read perf header" << endl;
+return 0;
+
+fail:
+cerr << errmsg;
+return -1;
}
int operf_read::get_eventnum_by_perf_event_id(u64 id) const
< at >< at > -507,32 +499,50 < at >< at > int operf_read::get_eventnum_by_perf_event_id(u64 id) const
return -1;
}
+int operf_read::_get_one_perf_event(event_t * event)
+{
+static size_t pe_header_size = sizeof(perf_event_header);
+char * evt = (char *)event;
+ssize_t num_read;
+perf_event_header * header = (perf_event_header *)event;
+errno = 0;
+if ((num_read = read(sample_data_fd, header, pe_header_size)) < 0) {
+cverb << vdebug << "Read 1 of sample data pipe returned with " << strerror(errno) << endl;
+return -1;
+} else if (num_read == 0) {
+return -1;
+}
+evt += pe_header_size;
+if (!header->size)
+return -1;
+if ((num_read = read(sample_data_fd, evt, header->size - pe_header_size)) < 0) {
+cverb << vdebug << "Read 2 of sample data pipe returned with " << strerror(errno) << endl;
+return -1;
+} else if (num_read == 0) {
+return -1;
+}
+return 0;
+}
+
int operf_read::convertPerfData(void)
{
int num_bytes = 0;
-struct mmap_info info;
-info.file_data_offset = opHeader.data_offset;
-info.file_data_size = opHeader.data_size;
-info.traceFD = open(inputFname.c_str(), O_RDONLY);
-if (op_mmap_trace_file(info) < 0) {
-close(info.traceFD);
-throw runtime_error("Error: Unable to mmap operf data file");
-}
+// Allocate way more than enough space for a really big event with a long callchain
+event_t * event = (event_t *)xmalloc(65536);
-cverb << vdebug << "Converting operf.data to oprofile sample data format" << endl;
-cverb << vdebug << "data size is " << hex << info.file_data_size << "; data offset is " << info.file_data_offset << endl;
-cverb << vdebug << "head is " << hex << info.head << endl;
+cverb << vdebug << "Converting operf data to oprofile sample data format" << endl;
cverb << vdebug << "sample type is " << hex << opHeader.h_attrs[0].attr.sample_type << endl;
first_time_processing = true;
+memset(event, '\0', 65536);
while (1) {
streamsize rec_size = 0;
-event_t * event = op_get_perf_event(info);
-if (event == NULL) {
+if (_get_one_perf_event(event) < 0) {
break;
}
rec_size = event->header.size;
-op_write_event(event, opHeader.h_attrs[0].attr.sample_type);
+if (op_write_event(event, opHeader.h_attrs[0].attr.sample_type) < 0)
+break;
num_bytes += rec_size;
}
first_time_processing = false;
< at >< at > -548,12 +558,12 < at >< at > int operf_read::convertPerfData(void)
delete images_it++->second;
all_images_map.clear();
-close(info.traceFD);
char * cbuf;
cbuf = (char *)xmalloc(operf_options::session_dir.length() + 5);
strcpy(cbuf, operf_options::session_dir.c_str());
strcat(cbuf, "/abi");
op_write_abi_to_file(cbuf);
free(cbuf);
+free(event);
return num_bytes;
}
diff --git a/libperf_events/operf_counter.h b/libperf_events/operf_counter.h
index 984fc01..ce1198e 100644
--- a/libperf_events/operf_counter.h
+++ b/libperf_events/operf_counter.h
< at >< at > -77,12 +77,12 < at >< at > public:
* For single app profiling, set sys_wide=false, the_pid=<processID-to-profile>,
* and pid_running=true if profiling an already active process; otherwise false.
*/
-operf_record(std::string outfile, bool sys_wide, pid_t the_pid, bool pid_running,
+operf_record(int output_pipe_fd, bool sys_wide, pid_t the_pid, bool pid_running,
std::vector<operf_event_t> & evts, OP_perf_utils::vmlinux_info_t vi,
bool callgraph, bool separate_by_cpu);
~operf_record();
void recordPerfData(void);
-int out_fd(void) const { return outputFile; }
+int out_fd(void) const { return output_fd; }
void add_to_total(int n) { total_bytes_recorded += n; }
int get_total_bytes_recorded(void) const { return total_bytes_recorded; }
void register_perf_event_id(unsigned counter, u64 id, perf_event_attr evt_attr);
< at >< at > -93,7 +93,7 < at >< at > private:
void setup(void);
int prepareToRecord(int counter, int cpu, int fd);
void write_op_header_info(void);
-int outputFile;
+int output_fd;
struct pollfd * poll_data;
std::vector< std::vector<struct mmap_data> > samples_array;
int num_cpus;
< at >< at > -115,7 +115,7 < at >< at > private:
class operf_read {
public:
operf_read(void) { valid = false; }
-void init(std::string infile, std::string samples_dir, op_cpu cputype,
+void init(int sample_data_pipe_fd, std::string samples_dir, op_cpu cputype,
std::vector<operf_event_t> & evts);
~operf_read();
int readPerfHeader(void);
< at >< at > -125,14 +125,14 < at >< at > public:
inline const operf_event_t * get_event_by_counter(u32 counter) { return &evts[counter]; }
private:
-std::string inputFname;
+int sample_data_fd;
std::string sampledir;
std::ifstream istrm;
struct OP_header opHeader;
std::vector<operf_event_t> evts;
bool valid;
op_cpu cpu_type;
-void read_op_header_info_with_ifstream(void);
+int _get_one_perf_event(event_t *);
};
diff --git a/libperf_events/operf_event.h b/libperf_events/operf_event.h
index 1513b0e..95feb9b 100644
--- a/libperf_events/operf_event.h
+++ b/libperf_events/operf_event.h
< at >< at > -107,12 +107,6 < at >< at > struct sample_data {
struct ip_callchain * callchain;
};
-struct mmap_info {
-u64 offset, file_data_size, file_data_offset, head;
-char * buf;
-int traceFD;
-};
-
typedef struct operf_event {
char name[OP_MAX_EVT_NAME_LEN];
< at >< at > -130,7 +124,6 < at >< at > typedef struct operf_event {
} operf_event_t;
struct op_file_section {
-u64 offset;
u64 size;
};
< at >< at > -142,7 +135,6 < at >< at > struct op_file_attr {
struct op_header_evt_info {
struct perf_event_attr attr;
std::vector<u64> ids;
-off_t id_offset;
};
struct OP_file_header {
< at >< at > -155,9 +147,6 < at >< at > struct OP_file_header {
struct OP_header {
struct op_header_evt_info h_attrs[OP_MAX_NUM_EVENTS];
-off_tattr_offset;
-off_tdata_offset;
-u64data_size;
};
/* Some of the above definitions were borrowed from the perf tool's util/event.h file. */
diff --git a/libperf_events/operf_sfile.cpp b/libperf_events/operf_sfile.cpp
index cc71439..ada47e1 100644
--- a/libperf_events/operf_sfile.cpp
+++ b/libperf_events/operf_sfile.cpp
< at >< at > -41,17 +41,7 < at >< at > static unsigned long
sfile_hash(struct operf_transient const * trans, struct operf_kernel_image * ki)
{
unsigned long val = 0;
-size_t fname_len = strlen(trans->image_name);
-/* fname_ptr will point at the first character in the image name
- * which we'll use for hashing. We don't need to hash on the whole
- * image name to get a decent hash. Arbitrarily, we'll hash on the
- * first 16 chars or fname_len (if fname_len < 16).
- */
-unsigned int fname_hash_len = (fname_len < 16) ? fname_len : 16;
-const char * fname_ptr = trans->image_name + fname_len - fname_hash_len;
-
-val ^= trans->tid << 2;
val ^= trans->tgid << 2;
if (operf_options::separate_cpu)
< at >< at > -61,12 +51,28 < at >< at > sfile_hash(struct operf_transient const * trans, struct operf_kernel_image * ki)
val ^= ki->start >> 14;
val ^= ki->end >> 7;
}
-for (unsigned int i = 0; i < fname_hash_len; i++)
-val = ((val << 5) + val) ^ fname_ptr[i];
if (trans->is_anon) {
val ^= trans->start_addr >> VMA_SHIFT;
val ^= trans->end_addr >> (VMA_SHIFT + 1);
+} else {
+size_t fname_len = trans->app_len;
+/* fname_ptr will point at the first character in the binary file's name
+ * which we'll use for hashing. We don't need to hash on the whole
+ * pathname to get a decent hash. Arbitrarily, we'll hash on the
+ * last 16 chars (or last fname_len chars if fname_len < 16).
+ */
+unsigned int fname_hash_len = (fname_len < 16) ? fname_len : 16;
+const char * fname_ptr = trans->app_filename + fname_len - fname_hash_len;
+for (unsigned int i = 0; i < fname_hash_len; i++)
+val = ((val << 5) + val) ^ fname_ptr[i];
+
+// Now do the same for image name
+fname_len = trans->image_len;
+fname_hash_len = (fname_len < 16) ? fname_len : 16;
+fname_ptr = trans->image_name + fname_len - fname_hash_len;
+for (unsigned int i = 0; i < fname_hash_len; i++)
+val = ((val << 5) + val) ^ fname_ptr[i];
}
return val & HASH_BITS;
< at >< at > -75,10 +81,11 < at >< at > sfile_hash(struct operf_transient const * trans, struct operf_kernel_image * ki)
static int
do_match(struct operf_sfile const * sf, struct operf_kernel_image const * ki,
- bool is_anon, const char * image_name, const char * appname, pid_t tgid,
- pid_t tid, unsigned int cpu)
+ bool is_anon, const char * image_name, size_t image_len,
+ const char * appname, size_t app_len,
+ pid_t tgid, pid_t tid, unsigned int cpu)
{
-size_t len1, len2;
+size_t shortest_image_len, shortest_app_len;
/* this is a simplified check for "is a kernel image" AND
* "is the right kernel image". Also handles no-vmlinux
< at >< at > -87,15 +94,13 < at >< at > do_match(struct operf_sfile const * sf, struct operf_kernel_image const * ki,
if (sf->kernel != ki)
return 0;
-if (sf->is_anon != is_anon)
+if (sf->tid != tid || sf->tgid != tgid)
return 0;
-len1 = strlen(sf->app_filename);
-len2 = strlen(appname);
-if ((len1 != len2) || strncmp(sf->app_filename, appname, len1))
+if (sf->is_anon != is_anon)
return 0;
-if (sf->tid != tid || sf->tgid != tgid)
+if ((sf->app_len != app_len) || (sf->image_len != image_len))
return 0;
if (operf_options::separate_cpu) {
< at >< at > -106,28 +111,22 < at >< at > do_match(struct operf_sfile const * sf, struct operf_kernel_image const * ki,
if (ki)
return 1;
-return (!strncmp(sf->image_name, image_name, len1));
+shortest_image_len = sf->image_len < image_len ? sf->image_len : image_len;
+if (strncmp(sf->image_name, image_name, shortest_image_len))
+return 0;
-}
-
-static int
-trans_match(struct operf_transient const * trans, struct operf_sfile const * sfile,
- struct operf_kernel_image const * ki)
-{
-return do_match(sfile, ki,
- trans->is_anon,
- trans->image_name, trans->app_filename,
- trans->tgid, trans->tid, trans->cpu);
+shortest_app_len = sf->app_len < app_len ? sf->app_len : app_len;
+return !strncmp(sf->app_filename, appname, shortest_app_len);
}
-
int
operf_sfile_equal(struct operf_sfile const * sf, struct operf_sfile const * sf2)
{
return do_match(sf, sf2->kernel,
sf2->is_anon,
- sf2->image_name, sf2->app_filename,
+ sf2->image_name, sf2->image_len,
+ sf2->app_filename, sf2->app_len,
sf2->tgid, sf2->tid, sf2->cpu);
}
< at >< at > -143,13 +142,14 < at >< at > create_sfile(unsigned long hash, struct operf_transient const * trans,
sf = (operf_sfile *)xmalloc(sizeof(struct operf_sfile));
sf->hashval = hash;
-
sf->tid = trans->tid;
sf->tgid = trans->tgid;
sf->cpu = 0;
sf->kernel = ki;
sf->image_name = trans->image_name;
sf->app_filename = trans->app_filename;
+sf->image_len = trans->image_len;
+sf->app_len = trans->app_len;
sf->is_anon = trans->is_anon;
sf->start_addr = trans->start_addr;
sf->end_addr = trans->end_addr;
< at >< at > -196,7 +196,11 < at >< at > struct operf_sfile * operf_sfile_find(struct operf_transient const * trans)
hash = sfile_hash(trans, ki);
list_for_each(pos, &hashes[hash]) {
sf = list_entry(pos, struct operf_sfile, hash);
-if (trans_match(trans, sf, ki)) {
+if (do_match(sf, ki,
+ trans->is_anon,
+ trans->image_name, trans->image_len,
+ trans->app_filename, trans->app_len,
+ trans->tgid, trans->tid, trans->cpu)) {
operf_sfile_get(sf);
goto lru;
}
diff --git a/libperf_events/operf_sfile.h b/libperf_events/operf_sfile.h
index 5ea02e5..9b91180 100644
--- a/libperf_events/operf_sfile.h
+++ b/libperf_events/operf_sfile.h
< at >< at > -41,6 +41,7 < at >< at > struct operf_sfile {
unsigned long hashval;
const char * image_name;
const char * app_filename;
+size_t image_len, app_len;
/** thread ID, -1 if not set */
pid_t tid;
/** thread group ID, -1 if not set */
< at >< at > -89,6 +90,7 < at >< at > struct operf_transient {
vma_t pc;
const char * image_name;
const char * app_filename;
+size_t image_len, app_len;
vma_t last_pc;
int event;
u64 sample_id;
diff --git a/libperf_events/operf_utils.cpp b/libperf_events/operf_utils.cpp
index 4ff99c0..42eecce 100644
--- a/libperf_events/operf_utils.cpp
+++ b/libperf_events/operf_utils.cpp
< at >< at > -38,6 +38,7 < at >< at >
extern verbose vmisc;
extern volatile bool quit;
+extern volatile bool read_quit;
extern operf_read operfRead;
extern int sample_reads;
extern unsigned int pagesize;
< at >< at > -161,22 +162,26 < at >< at > static void __handle_comm_event(event_t * event)
return;
}
#endif
-cverb << vperf << "PERF_RECORD_COMM for " << event->comm.comm << endl;
+if (cverb << vperf)
+cout << "PERF_RECORD_COMM for " << event->comm.comm << endl;
map<pid_t, operf_process_info *>::iterator it;
it = process_map.find(event->comm.pid);
if (it == process_map.end()) {
operf_process_info * proc = new operf_process_info(event->comm.pid,
app_name ? app_name : event->comm.comm,
app_name != NULL, true);
-cverb << vperf << "Adding new proc info to collection for PID " << event->comm.pid << endl;
+if (cverb << vperf)
+cout << "Adding new proc info to collection for PID " << event->comm.pid << endl;
process_map[event->comm.pid] = proc;
} else {
// sanity check -- should not get a second COMM event for same PID
if (it->second->is_valid()) {
-cverb << vperf << "Received extraneous COMM event for " << event->comm.comm
- << ", PID " << event->comm.pid << endl;
+if (cverb << vperf)
+cout << "Received extraneous COMM event for " << event->comm.comm
+ << ", PID " << event->comm.pid << endl;
} else {
-cverb << vperf << "Processing deferred mappings" << endl;
+if (cverb << vperf)
+cout << "Processing deferred mappings" << endl;
it->second->process_deferred_mappings(event->comm.comm);
}
}
< at >< at > -212,9 +217,11 < at >< at > static void __handle_mmap_event(event_t * event)
mapping->end_addr = (event->mmap.len == 0ULL)? 0ULL : mapping->start_addr + event->mmap.len - 1;
mapping->pgoff = event->mmap.pgoff;
-cverb << vperf << "PERF_RECORD_MMAP for " << event->mmap.filename << endl;
-cverb << vperf << "\tstart_addr: " << hex << mapping->start_addr;
-cverb << vperf << "; end addr: " << mapping->end_addr << endl;
+if (cverb << vperf) {
+cout << "PERF_RECORD_MMAP for " << event->mmap.filename << endl;
+cout << "\tstart_addr: " << hex << mapping->start_addr;
+cout << "; end addr: " << mapping->end_addr << endl;
+}
if (event->header.misc & PERF_RECORD_MISC_USER)
all_images_map.insert(pair<string, struct operf_mmap *>(image_basename, mapping));
< at >< at > -244,8 +251,9 < at >< at > static void __handle_mmap_event(event_t * event)
operf_process_info * proc = new operf_process_info(event->mmap.pid, app_name ? app_name : NULL,
app_name != NULL, false);
proc->add_deferred_mapping(mapping);
-cverb << vperf << "Added deferred mapping " << event->mmap.filename
-<< " for new process_info object" << endl;
+if (cverb << vperf)
+cout << "Added deferred mapping " << event->mmap.filename
+ << " for new process_info object" << endl;
process_map[event->mmap.pid] = proc;
#ifdef _TEST_DEFERRED_MAPPING
if (!do_comm_event) {
< at >< at > -255,8 +263,9 < at >< at > static void __handle_mmap_event(event_t * event)
#endif
} else if (!it->second->is_valid()) {
it->second->add_deferred_mapping(mapping);
-cverb << vperf << "Added deferred mapping " << event->mmap.filename
-<< " for existing but incomplete process_info object" << endl;
+if (cverb << vperf)
+cout << "Added deferred mapping " << event->mmap.filename
+ << " for existing but incomplete process_info object" << endl;
} else {
it->second->process_new_mapping(mapping);
}
< at >< at > -271,7 +280,8 < at >< at > static struct operf_transient * __get_operf_trans(struct sample_data * data)
if (trans.tgid == data->pid) {
proc = trans.cur_procinfo;
-cverb << vmisc << "trans.tgid == data->pid : " << data->pid << endl;
+if (cverb << vmisc)
+cout << "trans.tgid == data->pid : " << data->pid << endl;
} else {
// Find operf_process info for data.tgid.
< at >< at > -330,9 +340,12 < at >< at > static struct operf_transient * __get_operf_trans(struct sample_data * data)
op_mmap = proc->find_mapping_for_sample(data->ip);
}
if (op_mmap) {
-cverb << vperf << "Found mmap for sample; image_name is " << op_mmap->filename << endl;
+if (cverb << vperf)
+cout << "Found mmap for sample; image_name is " << op_mmap->filename << endl;
trans.image_name = op_mmap->filename;
trans.app_filename = proc->get_app_name().c_str();
+trans.image_len = strlen(trans.image_name);
+trans.app_len = strlen(trans.app_filename);
trans.start_addr = op_mmap->start_addr;
trans.end_addr = op_mmap->end_addr;
trans.tgid = data->pid;
< at >< at > -367,7 +380,8 < at >< at > static void __handle_callchain(u64 * array, struct sample_data * data)
{
data->callchain = (struct ip_callchain *) array;
if (data->callchain->nr) {
-cverb << vperf << "Processing callchain" << endl;
+if (cverb << vperf)
+cout << "Processing callchain" << endl;
for (int i = 0; i < data->callchain->nr; i++) {
data->ip = data->callchain->ips[i];
if (data->ip >= PERF_CONTEXT_MAX) {
< at >< at > -402,7 +416,7 < at >< at > static void __handle_callchain(u64 * array, struct sample_data * data)
static void __handle_sample_event(event_t * event, u64 sample_type)
{
struct sample_data data;
-bool early_match = false;
+bool found_trans = false;
const struct operf_mmap * op_mmap = NULL;
u64 *array = event->sample.array;
< at >< at > -457,9 +471,10 < at >< at > static void __handle_sample_event(event_t * event, u64 sample_type)
goto out;
}
-cverb << vperf << "(IP, " << event->header.misc << "): " << dec << data.pid << "/"
- << data.tid << ": " << hex << (unsigned long long)data.ip
- << endl << "\tdata ID: " << data.id << endl;
+if (cverb << vperf)
+cout << "(IP, " << event->header.misc << "): " << dec << data.pid << "/"
+ << data.tid << ": " << hex << (unsigned long long)data.ip
+ << endl << "\tdata ID: " << data.id << endl;
// Verify the sample.
trans.event = operfRead.get_eventnum_by_perf_event_id(data.id);
< at >< at > -471,42 +486,46 < at >< at > static void __handle_sample_event(event_t * event, u64 sample_type)
/* Check for the common case first -- i.e., where the current sample is from
* the same context as the previous sample. For the "no-vmlinux" case, start_addr
* and end_addr will be zero, so need to make sure we detect that.
+ * The last resort (and most expensive) is to call __get_operf_trans() if the
+ * sample cannot be matched up with a previous tran object.
*/
if (trans.in_kernel) {
if (trans.image_name && trans.tgid == data.pid) {
// For the no-vmlinux case . . .
if ((trans.start_addr == 0ULL) && (trans.end_addr == 0ULL)) {
trans.pc = data.ip;
-early_match = true;
+found_trans = true;
// For the case where a vmlinux file is passed in . . .
} else if (data.ip >= trans.start_addr && data.ip < trans.end_addr) {
trans.pc = data.ip;
-early_match = true;
+found_trans = true;
}
}
} else {
if (trans.tgid == data.pid && data.ip >= trans.start_addr && data.ip < trans.end_addr) {
trans.tid = data.tid;
trans.pc = data.ip - trans.start_addr;
-early_match = true;
+found_trans = true;
+} else if (__get_operf_trans(&data)) {
+trans.current = operf_sfile_find(&trans);
+found_trans = true;
}
+
}
-if (early_match || __get_operf_trans(&data)) {
-trans.current = operf_sfile_find(&trans);
-/*
- * trans.current may be NULL if a kernel sample falls through
- * the cracks, or if it's a sample from an anon region we couldn't find
- */
-if (trans.current) {
-/* log the sample or arc */
-operf_sfile_log_sample(&trans);
+/*
+ * trans.current may be NULL if a kernel sample falls through
+ * the cracks, or if it's a sample from an anon region we couldn't find
+ */
+if (found_trans && trans.current) {
+/* log the sample or arc */
+operf_sfile_log_sample(&trans);
-update_trans_last(&trans);
-if (sample_type & PERF_SAMPLE_CALLCHAIN)
-__handle_callchain(array, &data);
-goto out;
-}
+update_trans_last(&trans);
+if (sample_type & PERF_SAMPLE_CALLCHAIN)
+__handle_callchain(array, &data);
+goto out;
}
+
if (first_time_processing) {
event_t * ev = (event_t *)xmalloc(event->header.size);
memcpy(ev, event, event->header.size);
< at >< at > -519,17 +538,13 < at >< at > out:
/* This function is used by operf_read::convertPerfData() to convert perf-formatted
- * data to oprofile sample data files. After the header information in the perf data file,
- * the next piece of data is the PERF_RECORD_COMM record which tells us the name of the
+ * data to oprofile sample data files. After the header information in the perf sample data,
+ * the next piece of data is typically the PERF_RECORD_COMM record which tells us the name of the
* application/command being profiled. This is followed by PERF_RECORD_MMAP records
* which indicate what binary executables and libraries were mmap'ed into process memory
* when profiling began. Additional PERF_RECORD_MMAP records may appear later in the data
* stream (e.g., dlopen for single-process profiling or new process startup for system-wide
- * profiling. For such additional PERF_RECORD_MMAP records, it's possible that samples
- * may be collected and written to the perf data file for those mmap'ed binaries prior
- * to the reception/recording of the corresponding PERF_RECORD_MMAP record, in which case,
- * those samples are discarded and we increment the appropriate "lost sample" stat.
- * TODO: What's the name of the stat we increment?
+ * profiling.
*/
int OP_perf_utils::op_write_event(event_t * event, u64 sample_type)
{
< at >< at > -554,7 +569,6 < at >< at > int OP_perf_utils::op_write_event(event_t * event, u64 sample_type)
__handle_comm_event(event);
return 0;
case PERF_RECORD_EXIT:
-cverb << vperf << "PERF_RECORD_EXIT found in trace" << endl;
return 0;
default:
// OK, ignore all other header types.
< at >< at > -586,17 +600,11 < at >< at > void OP_perf_utils::op_perfrecord_sigusr1_handler(int sig __attribute__((unused)
quit = true;
}
-int OP_perf_utils::op_read_from_stream(ifstream & is, char * buf, streamsize sz)
+void OP_perf_utils::op_perfread_sigusr1_handler(int sig __attribute__((unused)),
+siginfo_t * siginfo __attribute__((unused)),
+void *u_context __attribute__((unused)))
{
-int rc = 0;
-is.read(buf, sz);
-if (!is.eof() && is.fail()) {
-cerr << "Internal error: Failed to read from input file." << endl;
-rc = -1;
-} else {
-rc = is.gcount();
-}
-return rc;
+read_quit = true;
}
int OP_perf_utils::op_write_output(int output, void *buf, size_t size)
< at >< at > -606,7 +614,7 < at >< at > int OP_perf_utils::op_write_output(int output, void *buf, size_t size)
int ret = write(output, buf, size);
if (ret < 0) {
-string errmsg = "Internal error: Failed to write to output file. errno is ";
+string errmsg = "Internal error: Failed to write sample data to pipe. errno is ";
errmsg += strerror(errno);
throw runtime_error(errmsg);
}
< at >< at > -670,7 +678,8 < at >< at > static void op_record_process_exec_mmaps(pid_t pid, pid_t tgid, int output_fd, o
mmap.header.size = (sizeof(mmap) -
(sizeof(mmap.filename) - size));
int num = OP_perf_utils::op_write_output(output_fd, &mmap, mmap.header.size);
-cverb << vperf << "Created MMAP event for " << imagename << endl;
+if (cverb << vperf)
+cout << "Created MMAP event for " << imagename << endl;
pr->add_to_total(num);
}
}
< at >< at > -769,11 +778,13 < at >< at > out:
op_record_process_exec_mmaps(pid, tgid, output_fd, pr);
fclose(fp);
-if (ret)
+if (ret) {
cverb << vperf << "couldn't get app name and tgid for pid "
<< dec << pid << " from /proc fs." << endl;
-else
-cverb << vperf << "Created COMM event for " << comm.comm << endl;
+} else {
+if (cverb << vperf)
+cout << "Created COMM event for " << comm.comm << endl;
+}
return ret;
}
< at >< at > -787,7 +798,8 < at >< at > int OP_perf_utils::op_record_process_info(bool system_wide, pid_t pid, operf_rec
int output_fd)
{
int ret;
-cverb << vperf << "op_record_process_info" << endl;
+if (cverb << vperf)
+cout << "op_record_process_info" << endl;
if (!system_wide) {
ret = _record_one_process_info(pid, system_wide, pr, output_fd);
} else {
< at >< at > -883,8 +895,9 < at >< at > static void _record_module_info(int output_fd, operf_record * pr)
mmap.header.size = (sizeof(mmap) -
(sizeof(mmap.filename) - size));
int num = OP_perf_utils::op_write_output(output_fd, &mmap, mmap.header.size);
-cverb << vperf << "Created MMAP event for " << module_name << ". Size: "
- << module_size << "; start addr: " << start_address << endl;
+if (cverb << vperf)
+cout << "Created MMAP event for " << module_name << ". Size: "
+ << module_size << "; start addr: " << start_address << endl;
pr->add_to_total(num);
free(line);
}
< at >< at > -917,8 +930,9 < at >< at > void OP_perf_utils::op_record_kernel_info(string vmlinux_file, u64 start_addr, u
mmap.header.size = (sizeof(mmap) -
(sizeof(mmap.filename) - size));
int num = op_write_output(output_fd, &mmap, mmap.header.size);
-cverb << vperf << "Created MMAP event for " <<mmap.filename << ". Size: "
- << mmap.len << "; start addr: " << mmap.start << endl;
+if (cverb << vperf)
+cout << "Created MMAP event of size " << mmap.header.size << " for " <<mmap.filename << ". length: "
+ << mmap.len << "; start addr: " << mmap.start << endl;
pr->add_to_total(num);
_record_module_info(output_fd, pr);
}
< at >< at > -965,101 +979,6 < at >< at > void OP_perf_utils::op_get_kernel_event_data(struct mmap_data *md, operf_record
}
-static size_t mmap_size;
-static size_t pg_sz;
-
-static int __mmap_trace_file(struct mmap_info & info)
-{
-int mmap_prot = PROT_READ;
-int mmap_flags = MAP_SHARED;
-
-info.buf = (char *) mmap(NULL, mmap_size, mmap_prot,
-mmap_flags, info.traceFD, info.offset);
-if (info.buf == MAP_FAILED) {
-cerr << "Error: mmap failed with errno:\n\t" << strerror(errno) << endl;
-return -1;
-}
-else {
-cverb << vperf << hex << "mmap with the following parameters" << endl
- << "\tinfo.head: " << info.head << endl
- << "\tinfo.offset: " << info.offset << endl;
-return 0;
-}
-}
-
-int OP_perf_utils::op_mmap_trace_file(struct mmap_info & info)
-{
-u64 shift;
-if (!pg_sz)
-pg_sz = sysconf(_SC_PAGESIZE);
-if (!mmap_size) {
-if (MMAP_WINDOW_SZ > info.file_data_size) {
-mmap_size = info.file_data_size;
-} else {
-mmap_size = MMAP_WINDOW_SZ;
-}
-}
-info.offset = 0;
-info.head = info.file_data_offset;
-shift = pg_sz * (info.head / pg_sz);
-info.offset += shift;
-info.head -= shift;
-return __mmap_trace_file(info);
-}
-
-event_t * OP_perf_utils::op_get_perf_event(struct mmap_info & info)
-{
-uint32_t size;
-event_t * event;
-
-if (info.offset + info.head >= info.file_data_offset + info.file_data_size)
-return NULL;
-
-if (!pg_sz)
-pg_sz = sysconf(_SC_PAGESIZE);
-
-try_again:
-event = (event_t *)(info.buf + info.head);
-
-if ((mmap_size != info.file_data_size) &&
-(((info.head + sizeof(event->header)) > mmap_size) ||
-(info.head + event->header.size > mmap_size))) {
-int ret;
-u64 shift = pg_sz * (info.head / pg_sz);
-cverb << vperf << "Remapping perf data file" << endl;
-ret = munmap(info.buf, mmap_size);
-if (ret) {
-string errmsg = "Internal error: munmap of perf data file failed with errno: ";
-errmsg += strerror(errno);
-throw runtime_error(errmsg);
-}
-
-info.offset += shift;
-info.head -= shift;
-ret = __mmap_trace_file(info);
-if (ret) {
-string errmsg = "Internal error: mmap of perf data file failed with errno: ";
-errmsg += strerror(errno);
-throw runtime_error(errmsg);
-}
-goto try_again;
-}
-
-size = event->header.size;
-
-// The tail end of the operf data file may be zero'ed out, so we assume if we
-// find size==0, we're now in that area of the file, so we're done.
-if (size == 0)
-return NULL;
-
-info.head += size;
-if (info.offset + info.head >= info.file_data_offset + info.file_data_size)
-return NULL;
-
-return event;
-}
-
-
int OP_perf_utils::op_get_next_online_cpu(DIR * dir, struct dirent *entry)
{
#define OFFLINE 0x30
diff --git a/libperf_events/operf_utils.h b/libperf_events/operf_utils.h
index 060ee2b..9359c82 100644
--- a/libperf_events/operf_utils.h
+++ b/libperf_events/operf_utils.h
< at >< at > -38,12 +38,6 < at >< at > extern bool no_vmlinux;
#define OP_APPNAME_LEN 1024
-#if BITS_PER_LONG == 64
-#define MMAP_WINDOW_SZ ULLONG_MAX
-#else
-#define MMAP_WINDOW_SZ (32 * 1024 * 1024ULL)
-#endif
-
extern unsigned int op_nr_counters;
static inline size_t align_64bit(u64 x)
< at >< at > -64,12 +58,12 < at >< at > void op_get_kernel_event_data(struct mmap_data *md, operf_record * pr);
void op_perfrecord_sigusr1_handler(int sig __attribute__((unused)),
siginfo_t * siginfo __attribute__((unused)),
void *u_context __attribute__((unused)));
+void op_perfread_sigusr1_handler(int sig __attribute__((unused)),
+siginfo_t * siginfo __attribute__((unused)),
+void *u_context __attribute__((unused)));
int op_record_process_info(bool system_wide, pid_t pid, operf_record * pr, int output_fd);
int op_write_output(int output, void *buf, size_t size);
int op_write_event(event_t * event, u64 sample_type);
-int op_read_from_stream(std::ifstream & is, char * buf, std::streamsize sz);
-int op_mmap_trace_file(struct mmap_info & info);
-event_t * op_get_perf_event(struct mmap_info & info);
int op_get_next_online_cpu(DIR * dir, struct dirent *entry);
bool op_convert_event_vals(std::vector<operf_event_t> * evt_vec);
void op_reprocess_unresolved_events(u64 sample_type);
diff --git a/pe_profiling/operf.cpp b/pe_profiling/operf.cpp
index 95bf183..ee4e3ba 100644
--- a/pe_profiling/operf.cpp
+++ b/pe_profiling/operf.cpp
< at >< at > -47,8 +47,10 < at >< at > using namespace std;
typedef enum END_CODE {
ALL_OK = 0,
-APP_ABNORMAL_END = -2,
-PERF_RECORD_ERROR = -3
+APP_ABNORMAL_END = 1,
+PERF_RECORD_ERROR = 2,
+PERF_READ_ERROR = 4,
+PERF_BOTH_ERROR = 8
} end_code_t;
// Globals
< at >< at > -60,9 +62,10 < at >< at > double cpu_speed;
char op_samples_current_dir[PATH_MAX];
uint op_nr_counters;
verbose vmisc("misc");
+static void convert_sample_data(void);
+static int sample_data_pipe[2];
-#define DEFAULT_OPERF_OUTFILE "operf.data"
#define CALLGRAPH_MIN_COUNT_SCALE 15
static char full_pathname[PATH_MAX];
< at >< at > -72,10 +75,9 < at >< at > static pid_t app_PID = -1;
static pid_t jitconv_pid = -1;
static bool app_started;
static pid_t operf_pid;
+static pid_t convert_pid;
static string samples_dir;
-static string outputfile;
static bool startApp;
-static bool reset_done = false;
static char start_time_str[32];
static vector<operf_event_t> events;
static bool jit_conversion_running;
< at >< at > -255,6 +257,8 < at >< at > int start_profiling_app(void)
perror("Internal error: fork failed");
_exit(EXIT_FAILURE);
} else if (app_PID == 0) { // child process for exec'ing app
+close(sample_data_pipe[0]);
+close(sample_data_pipe[1]);
run_app();
}
// parent
< at >< at > -270,6 +274,7 < at >< at > int start_profiling_app(void)
return -1;
} else if (operf_pid == 0) { // operf-record process
int ready = 0;
+close(sample_data_pipe[0]);
/*
* Since an informative message will be displayed to the user if
* an error occurs, we don't want to blow chunks here; instead, we'll
< at >< at > -281,7 +286,7 < at >< at > int start_profiling_app(void)
vi.image_name = operf_options::vmlinux;
vi.start = kernel_start;
vi.end = kernel_end;
-operf_record operfRecord(outputfile, operf_options::system_wide, app_PID,
+operf_record operfRecord(sample_data_pipe[1], operf_options::system_wide, app_PID,
(operf_options::pid == app_PID), events, vi,
operf_options::callgraph,
operf_options::separate_cpu);
< at >< at > -308,7 +313,7 < at >< at > int start_profiling_app(void)
// start recording
operfRecord.recordPerfData();
-cerr << "Total bytes recorded from perf events: "
+cverb << vmisc << "Total bytes recorded from perf events: " << dec
<< operfRecord.get_total_bytes_recorded() << endl;
operfRecord.~operf_record();
< at >< at > -319,9 +324,6 < at >< at > int start_profiling_app(void)
goto fail_out;
}
fail_out:
-ofstream of;
-of.open(outputfile.c_str(), ios_base::trunc);
-of.close();
if (startApp && !ready){
/* ready==0 means we've not yet told parent we're ready,
* but the parent is reading our pipe. So we tell the
< at >< at > -376,6 +378,10 < at >< at > static end_code_t _kill_operf_pid(void)
{
int waitpid_status = 0;
end_code_t rc = ALL_OK;
+struct timeval tv;
+long long start_time_sec;
+long long usec_timer;
+bool keep_trying = true;
// stop operf-record process
if (kill(operf_pid, SIGUSR1) < 0) {
< at >< at > -387,14 +393,67 < at >< at > static end_code_t _kill_operf_pid(void)
rc = PERF_RECORD_ERROR;
} else {
if (WIFEXITED(waitpid_status) && (!WEXITSTATUS(waitpid_status))) {
-cverb << vdebug << "waitpid for operf-record process returned OK" << endl;
+cverb << vdebug << "operf-record process returned OK" << endl;
} else {
cerr << "operf-record process ended abnormally: "
-<< WEXITSTATUS(waitpid_status) << endl;
+ << WEXITSTATUS(waitpid_status) << endl;
rc = PERF_RECORD_ERROR;
}
}
}
+
+// Now stop the operf-read process (aka "convert_pid")
+waitpid_status = 0;
+gettimeofday(&tv, NULL);
+start_time_sec = tv.tv_sec;
+usec_timer = tv.tv_usec;
+/* We'll initially try the waitpid with WNOHANG once every 100,000 usecs.
+ * If it hasn't ended within 5 seconds, we'll kill it and do one
+ * final wait.
+ */
+while (keep_trying) {
+int option = WNOHANG;
+gettimeofday(&tv, NULL);
+if (tv.tv_sec > start_time_sec + 5) {
+keep_trying = false;
+option = 0;
+cerr << "now trying to kill convert pid..." << endl;
+
+if (kill(convert_pid, SIGUSR1) < 0) {
+perror("Attempt to stop operf-read process failed");
+rc = rc ? PERF_BOTH_ERROR : PERF_READ_ERROR;
+break;
+}
+} else {
+/* If we exceed the 100000 usec interval or if the tv_usec
+ * value has rolled over to restart at 0, then we reset
+ * the usec_timer to current tv_usec and try waitpid.
+ */
+if ((tv.tv_usec % 1000000) > (usec_timer + 100000)
+|| (tv.tv_usec < usec_timer))
+usec_timer = tv.tv_usec;
+else
+continue;
+}
+if (waitpid(convert_pid, &waitpid_status, option) < 0) {
+keep_trying = false;
+if (errno != ECHILD) {
+perror("waitpid for operf-read process failed");
+rc = rc ? PERF_BOTH_ERROR : PERF_READ_ERROR;
+}
+} else {
+if (WIFEXITED(waitpid_status)) {
+keep_trying = false;
+if (!WEXITSTATUS(waitpid_status)) {
+cverb << vdebug << "operf-read process returned OK" << endl;
+} else {
+cerr << "operf-read process ended abnormally. Status = "
+ << WEXITSTATUS(waitpid_status) << endl;
+rc = rc ? PERF_BOTH_ERROR : PERF_READ_ERROR;
+}
+}
+}
+}
return rc;
}
< at >< at > -408,13 +467,40 < at >< at > static end_code_t _run(void)
sigfillset(&ss);
sigprocmask(SIG_BLOCK, &ss, NULL);
+// Create pipe to which operf-record process writes sample data and
+// from which the operf-read process reads.
+if (pipe(sample_data_pipe) < 0) {
+perror("Internal error: operf-record could not create pipe");
+_exit(EXIT_FAILURE);
+}
+
if (start_profiling_app() < 0) {
return PERF_RECORD_ERROR;
}
// parent continues here
if (startApp)
cverb << vdebug << "app " << app_PID << " is running" << endl;
+
+/* If we're not doing system wide profiling and no app is started, then
+ * there's no profile data to convert. So if this condition is NOT true,
+ * then we'll do the convert.
+ */
+if (!(!app_started && !operf_options::system_wide)) {
+convert_pid = fork();
+if (convert_pid < 0) {
+perror("Internal error: fork failed");
+_exit(EXIT_FAILURE);
+} else if (convert_pid == 0) { // child process
+close(sample_data_pipe[1]);
+convert_sample_data();
+}
+// parent
+close(sample_data_pipe[0]);
+close(sample_data_pipe[1]);
+}
+
set_signals();
+cout << "operf: Profiler started" << endl;
if (startApp) {
// User passed in command or program name to start
cverb << vdebug << "going into waitpid on profiled app " << app_PID << endl;
< at >< at > -438,6 +524,7 < at >< at > static end_code_t _run(void)
rc = _kill_operf_pid();
} else {
// User passed in --pid or --system-wide
+cout << "operf: Press Ctl-c to stop profiling" << endl;
cverb << vdebug << "going into waitpid on operf record process " << operf_pid << endl;
if (waitpid(operf_pid, &waitpid_status, 0) < 0) {
if (errno == EINTR) {
< at >< at > -453,7 +540,7 < at >< at > static end_code_t _run(void)
cverb << vdebug << "waitpid for operf-record process returned OK" << endl;
} else if (WIFEXITED(waitpid_status)) {
cerr << "operf-record process ended abnormally: "
-<< WEXITSTATUS(waitpid_status) << endl;
+ << WEXITSTATUS(waitpid_status) << endl;
rc = PERF_RECORD_ERROR;
} else if (WIFSIGNALED(waitpid_status)) {
cerr << "operf-record process killed by signal "
< at >< at > -467,12 +554,10 < at >< at > static end_code_t _run(void)
static void cleanup(void)
{
-string cmd = "rm -f " + outputfile;
free(app_name_SAVE);
free(app_args);
events.clear();
verbose_string.clear();
-system(cmd.c_str());
}
static void _jitconv_complete(int val __attribute__((unused)))
< at >< at > -572,17 +657,17 < at >< at > static int __delete_old_previous_sample_data(const char *fpath,
}
}
-static void complete(void)
+/* Read perf_events sample data written by the operf-record process
+ * through the sample_data_pipe and convert this to oprofile format
+ * sample files.
+ */
+static void convert_sample_data(void)
{
int rc;
string current_sampledir = samples_dir + "/current/";
string previous_sampledir = samples_dir + "/previous";
current_sampledir.copy(op_samples_current_dir, current_sampledir.length(), 0);
-if (!app_started && !operf_options::system_wide) {
-cleanup();
-return;
-}
if (!operf_options::append) {
int flags = FTW_DEPTH | FTW_ACTIONRETVAL;
errno = 0;
< at >< at > -592,47 +677,40 < at >< at > static void complete(void)
if (errno)
cerr << strerror(errno) << endl;
cleanup();
-exit(1);
+_exit(EXIT_FAILURE);
}
if (rename(current_sampledir.c_str(), previous_sampledir.c_str()) < 0) {
if (errno && (errno != ENOENT)) {
cerr << "Unable to move old profile data to " << previous_sampledir << endl;
cerr << strerror(errno) << endl;
cleanup();
-exit(1);
+_exit(EXIT_FAILURE);
}
}
-reset_done = true;
}
rc = mkdir(current_sampledir.c_str(), S_IRWXU | S_IRWXG | S_IROTH | S_IXOTH);
if (rc && (errno != EEXIST)) {
cerr << "Error trying to create " << current_sampledir << " dir." << endl;
perror("mkdir failed with");
-exit(EXIT_FAILURE);
+_exit(EXIT_FAILURE);
}
-operfRead.init(outputfile, current_sampledir, cpu_type, events);
+operfRead.init(sample_data_pipe[0], current_sampledir, cpu_type, events);
if ((rc = operfRead.readPerfHeader()) < 0) {
if (rc != OP_PERF_HANDLED_ERROR)
-cerr << "Error: Cannot create read header info for sample file " << outputfile << endl;
+cerr << "Error: Cannot create read header info for sample data " << endl;
cleanup();
-exit(1);
+_exit(EXIT_FAILURE);
}
-cverb << vdebug << "Successfully read header info for sample file " << outputfile << endl;
-// TODO: We may want to do incremental conversion of the perf data, since the perf sample format
-// is very inefficient to store. For example, using a simple test program that does many
-// millions of memcpy's over a 12 second span of time, a profile taken via legacy oprofile,
-// with --separate=all and --image=<app_name> requires ~300K of storage. Using the perf tool
-// (not operf) to profile the same application creates an 18MB perf.data file!!
+cverb << vdebug << "Successfully read header info for sample data " << endl;
if (operfRead.is_valid()) {
try {
operfRead.convertPerfData();
-cerr << endl << "Use '--session-dir=" << operf_options::session_dir << "'" << endl
- << "with opreport and other post-processing tools to view your profile data."
- << endl;
} catch (runtime_error e) {
cerr << "Caught exception from operf_read::convertPerfData" << endl;
cerr << e.what() << endl;
+cleanup();
+_exit(EXIT_FAILURE);
}
}
// Invoke opjitconv and set up a SIGCHLD signal for when it's done
< at >< at > -645,7 +723,8 < at >< at > static void complete(void)
if (jit_conversion_running) {
kill(jitconv_pid, SIGKILL);
}
-cleanup();
+_exit(EXIT_SUCCESS);
+
}
< at >< at > -1072,7 +1151,6 < at >< at > static void process_args(int argc, char const ** argv)
}
_process_session_dir();
-outputfile = samples_dir + "/" + DEFAULT_OPERF_OUTFILE;
if (operf_options::evts.empty()) {
// Use default event
< at >< at > -1137,6 +1215,7 < at >< at > static void _precheck_permissions_to_samplesdir(string sampledir, bool for_curre
cleanup();
exit(1);
}
+afile.close();
}
< at >< at > -1200,13 +1279,18 < at >< at > int main(int argc, char const *argv[])
perror("Attempt to kill profiled app failed.");
}
}
-if (run_result == PERF_RECORD_ERROR) {
+if ((run_result == PERF_RECORD_ERROR) || (run_result == PERF_BOTH_ERROR)) {
cerr << "Error running profiler" << endl;
-exit(1);
+} else if (run_result == PERF_READ_ERROR) {
+cerr << "Error converting operf sample data to oprofile sample format" << endl;
} else {
cerr << "WARNING: Profile results may be incomplete due to to abend of profiled app." << endl;
}
+} else {
+cout << endl << "Use '--session-dir=" << operf_options::session_dir << "'" << endl
+ << "with opreport and other post-processing tools to view your profile data."
+ << endl;
}
-complete();
-return 0;
+cleanup();
+return run_result;;
}
</pre>Maynard Johnson2012-05-22T22:05:40[PATCH] Remove reset option from operf and replace with append option
http://comments.gmane.org/gmane.linux.oprofile/10503
<pre>[PATCH] Remove reset option from operf and replace with append option.
By default, operf will now move old profile data from <session_dir>/samples/current
to <session_dir>/samples/previous. If a 'previous' profile already existed,
it will be replaced. If the --append option is passed, old profile data is left
in place and new profile data will be added to it, and the 'previous' profile
(if one existed) will remain untouched.
operf man page updates:
Aside from replacing the --reset option with the --append option, several unrelated
fixes were made, including adding the missing --vmlinux option.
This patch has been pushed up to the perf-events branch, but review comments are
as always, welcome.
Signed-off-by: Maynard Johnson <maynardj< at >us.ibm.com>
---
doc/operf.1.in | 36 ++++++++++++----
pe_profiling/operf.cpp | 109 +++++++++++++++++++++++++++--------------------
2 files changed, 90 insertions(+), 55 deletions(-)
diff --git a/doc/operf.1.in b/doc/operf.1.in
index 00b6b2c..00fb5f3 100644
--- a/doc/operf.1.in
+++ b/doc/operf.1.in
< at >< at > -44,8 +44,9 < at >< at > is required, but
be used simultaneously.
.br
.TP
-.BI "--pid / -p [PID]"
-This option enables operf to profile a running application. <PID>
+.BI "--pid / -p " PID
+This option enables operf to profile a running application.
+.I PID
should be the process ID of the process you wish to profile. When
finished profiling (e.g., when the profiled process ends), press
Ctrl-c to stop operf.
< at >< at > -71,21 +72,38 < at >< at > directory should be /root or a subdirectory of /root to avoid
storing sample data files in locations accessible by regular users.
.br
.TP
+.BI "--vmlinux / k " vmlinux_path
+A vmlinux file that matches the running kernel that has symbol and/or debuginfo.
+Kernel samples will be attributed to this binary, allowing post-processing tools
+(like opreport) to attribute samples to the appropriate kernel symbols.
+.TP
.BI "--callgraph / -g"
This option enables the callgraph to be saved during profiling. NOTE: The
full callchain is recorded, so there is no depth limit.
.br
.TP
-.BI "--reset / -r"
-This option clears out old profile data. You should almost always do this.
+.BI "--append / -a"
+By default,
+.I operf
+moves old profile data from <session_dir>/samples/current to
+<session_dir>/samples/previous. If a 'previous' profile already existed,
+it will be replaced. If the
+.I --append
+option is passed, old profile data is left in place and new profile data will
+be added to it, and the 'previous' profile (if one existed) will remain untouched.
+To access the 'previous' profile, simply add a session specification to the normal
+invocation of oprofile post-processing tools. For example:
+.br
+.I " opreport --session-dir=`pwd`/oprofile_data session:previous"
.br
.TP
-.BI "--kernel-buffersize-multiplier / -k [buffersize]"
+.BI "--kernel-buffersize-multiplier / -k " buffersize
This option enables operf to increase the kernel buffer size by a
-factor of <buffersize>.
+factor of
+.I buffersize.
.br
.TP
-.BI "--events / -e [event1[,event2[,...]]]"
+.BI "--events / -e " event1[,event2[,...]]
This option is for passing a comma-separated list of event specifications
for profiling. Each event spec is of the form:
.br
< at >< at > -108,14 +126,14 < at >< at > executing the same program during a profiling run.
This option categorizes samples by cpu.
.br
.TP
-.BI "--session-dir [path]"
+.BI "--session-dir / -d " path
This option specifies the session path to hold the sample data. If not specified,
the data is saved in the
.I oprofile_data
directory on the current path.
.br
.TP
-.BI "--verbose / -V [level]"
+.BI "--verbose / -V " level
A comma-separated list of debugging control values, used to increase the verbosity of the output.
Valid values are: debug, perf_events, misc, sfile, arcs, or the special value, 'all'.
.br
diff --git a/pe_profiling/operf.cpp b/pe_profiling/operf.cpp
index 4b2c65f..95bf183 100644
--- a/pe_profiling/operf.cpp
+++ b/pe_profiling/operf.cpp
< at >< at > -84,7 +84,7 < at >< at > static uid_t my_uid;
namespace operf_options {
bool system_wide;
-bool reset;
+bool append;
int pid;
bool callgraph;
int mmap_pages_mult;
< at >< at > -109,8 +109,8 < at >< at > popt::option options_array[] = {
"enable callgraph recording"),
popt::option(operf_options::system_wide, "system-wide", 's',
"profile entire system"),
-popt::option(operf_options::reset, "reset", 'r',
- "clear out old profile data"),
+popt::option(operf_options::append, "append", 'a',
+ "add new profile data to old profile data"),
popt::option(operf_options::pid, "pid", 'p',
"process ID to profile", "PID"),
popt::option(operf_options::mmap_pages_mult, "kernel-buffersize-multiplier", 'b',
< at >< at > -475,19 +475,6 < at >< at > static void cleanup(void)
system(cmd.c_str());
}
-static int __delete_sample_data(const char *fpath,
- const struct stat *sb __attribute__((unused)),
- int tflag __attribute__((unused)),
- struct FTW *ftwbuf __attribute__((unused)))
-{
-if (remove(fpath)) {
-perror("sample data removal error");
-return FTW_STOP;
-} else {
-return FTW_CONTINUE;
-}
-}
-
static void _jitconv_complete(int val __attribute__((unused)))
{
int child_status;
< at >< at > -572,29 +559,49 < at >< at > static void _do_jitdump_convert()
}
+static int __delete_old_previous_sample_data(const char *fpath,
+ const struct stat *sb __attribute__((unused)),
+ int tflag __attribute__((unused)),
+ struct FTW *ftwbuf __attribute__((unused)))
+{
+if (remove(fpath)) {
+perror("sample data removal error");
+return FTW_STOP;
+} else {
+return FTW_CONTINUE;
+}
+}
static void complete(void)
{
int rc;
string current_sampledir = samples_dir + "/current/";
+string previous_sampledir = samples_dir + "/previous";
current_sampledir.copy(op_samples_current_dir, current_sampledir.length(), 0);
if (!app_started && !operf_options::system_wide) {
cleanup();
return;
}
-if (operf_options::reset) {
-int flags = FTW_DEPTH | FTW_ACTIONRETVAL;
+if (!operf_options::append) {
+ int flags = FTW_DEPTH | FTW_ACTIONRETVAL;
errno = 0;
-if (nftw(current_sampledir.c_str(), __delete_sample_data, 32, flags) !=0 &&
+if (nftw(previous_sampledir.c_str(), __delete_old_previous_sample_data, 32, flags) !=0 &&
errno != ENOENT) {
-cerr << "Unable to remove old sample data at "
- << current_sampledir << "." << endl;
+cerr << "Unable to remove old sample data at " << previous_sampledir << "." << endl;
if (errno)
cerr << strerror(errno) << endl;
cleanup();
exit(1);
}
+if (rename(current_sampledir.c_str(), previous_sampledir.c_str()) < 0) {
+if (errno && (errno != ENOENT)) {
+cerr << "Unable to move old profile data to " << previous_sampledir << endl;
+cerr << strerror(errno) << endl;
+cleanup();
+exit(1);
+}
+}
reset_done = true;
}
rc = mkdir(current_sampledir.c_str(), S_IRWXU | S_IRWXG | S_IROTH | S_IXOTH);
< at >< at > -1104,6 +1111,35 < at >< at > static int _check_perf_events_cap(void)
}
+static void _precheck_permissions_to_samplesdir(string sampledir, bool for_current)
+{
+/* Pre-check to make sure we have permission to remove old sample data
+ * or to create new sample data in the specified sample data directory.
+ * If the user wants us to remove old data, we don't actually do it now,
+ * since the profile session may fail for some reason or the user may do ctl-c.
+ * We should exit without unnecessarily removing the old sample data as
+ * the user may expect it to still be there after an aborted run.
+ */
+string sampledir_testfile = sampledir + "/.xxxTeStFiLe";
+ofstream afile;
+errno = 0;
+afile.open(sampledir_testfile.c_str());
+if (!afile.is_open() && (errno != ENOENT)) {
+if (operf_options::append && for_current)
+cerr << "Unable to write to sample data directory at "
+ << sampledir << "." << endl;
+else
+cerr << "Unable to remove old sample data at "
+ << sampledir << "." << endl;
+if (errno)
+cerr << strerror(errno) << endl;
+cerr << "Try a manual removal of " << sampledir << endl;
+cleanup();
+exit(1);
+}
+
+}
+
bool no_vmlinux;
int main(int argc, char const *argv[])
{
< at >< at > -1140,30 +1176,13 < at >< at > int main(int argc, char const *argv[])
op_nr_counters = op_get_nr_counters(cpu_type);
if (my_uid != 0) {
-/* Pre-check to make sure we have permission to remove old sample data
- * or to create new sample data in the specified sample data directory.
- * If the user wants us to remove old data, we don't actually do it now,
- * since the profile session may fail for some reason or the user may do ctl-c.
- * We should exit without unnecessarily removing the old sample data as
- * the user may expect it to still be there after an aborted run.
- */
+bool for_current = true;
string current_sampledir = samples_dir + "/current";
-string current_sampledir_testfile = current_sampledir + "/.xxxTeStFiLe";
-ofstream afile;
-errno = 0;
-afile.open(current_sampledir_testfile.c_str());
-if (!afile.is_open() && (errno != ENOENT)) {
-if (operf_options::reset)
-cerr << "Unable to remove old sample data at "
- << current_sampledir << "." << endl;
-else
-cerr << "Unable to write to sample data directory at "
- << current_sampledir << "." << endl;
-if (errno)
-cerr << strerror(errno) << endl;
-cerr << "Try a manual removal of " << current_sampledir << endl;
-cleanup();
-exit(1);
+_precheck_permissions_to_samplesdir(current_sampledir, for_current);
+if (!operf_options::append) {
+string previous_sampledir = samples_dir + "/previous";
+for_current = false;
+_precheck_permissions_to_samplesdir(previous_sampledir, for_current);
}
}
end_code_t run_result;
< at >< at > -1189,7 +1208,5 < at >< at > int main(int argc, char const *argv[])
}
}
complete();
-if (operf_options::reset && reset_done == false)
-cerr << "Requested reset was not performed due to problem running operf command." << endl;
return 0;
}
</pre>Maynard Johnson2012-05-22T14:15:32[ oprofile-Bugs-3528383 ] oprofied failed to open device on mpc8572linux version 2.6.
http://comments.gmane.org/gmane.linux.oprofile/10498
<pre>Bugs item #3528383, was opened at 2012-05-20 04:43
Message generated for change (Comment added) made by maynardj
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=116191&aid=3528383&group_id=16191
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Priority: 5
Private: No
Submitted By: tombkeeper (tombkeeper)
Assigned to: Nobody/Anonymous (nobody)
Summary: oprofied failed to open device on mpc8572 linux version 2.6.
Initial Comment:
Linux version: 2.6.31
CPU: mpc8572, e500 core
oprofile version: 0.9.7
1>oprofile --init is ok
2>oprofile --start failed, error output:
Using 2.6+ OProfile kernel interface.
Failed to open device. Possibly you have passed incorrect
parameters. Check /var/log/messages.Couldn't start oprofiled.
Check the log file "/var/lib/oprofile/samples/oprofiled.log" and kernel syslog
3>check /var/log/message, nothing there. check oprofiled.log, no such file exist.
4>oprofile -l: list all features supported by e500
I tried to use strace to find which open failed and got lots of /dev/oprofile/* opened, not sure which failed. Also walking through oprofile code is not helpful, only see opening /dev/oprofile/buffer code will output such errors.
Anyone knows how to debug this, to figure out if the issue is in kernel or application, if it is some incorrect parameter?
Thanks.
John
----------------------------------------------------------------------
Date: 2012-05-21 05:00
Message:
Opening a bug report for situations like this won't help you. You need to
ask for help from the community by posting a message to the list and/or
using #oprofile IRC on OFTC.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=116191&aid=3528383&group_id=16191
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
</pre>SourceForge.net2012-05-21T12:00:35Getting Data from Oprofile within JVM
http://comments.gmane.org/gmane.linux.oprofile/10496
<pre>What would be a good place to start to be able to get data from oprofile
within JVM. i.e. i want to have JVM spawn a thread running the task of
oprofiled. The data is then passed to and processed by the thread.
Thanks
Xin
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
oprofile-list mailing list
oprofile-list< at >lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oprofile-list
</pre>Xin Tong2012-05-21T04:16:15BR_INST_RETIRED on Intel microarchitecture
http://comments.gmane.org/gmane.linux.oprofile/10495
<pre>I am using oprofile in a JVM environment. I have 2 questions
1. I am not very familiar with intel microarchitecrure, BR_INST_RETIRED
means the branch decision is correct and the branch instruction is
committed ?
2. Can one obtain the address branched to in the PMU registers ? Oprofile
does not seem to provide it.
Thanks
Xin
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
oprofile-list mailing list
oprofile-list< at >lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oprofile-list
</pre>Xin Tong2012-05-20T17:28:28[ oprofile-Bugs-3528383 ] oprofied failed to open device on mpc8572linux version 2.6.
http://comments.gmane.org/gmane.linux.oprofile/10494
<pre>Bugs item #3528383, was opened at 2012-05-20 04:43
Message generated for change (Tracker Item Submitted) made by tombkeeper
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=116191&aid=3528383&group_id=16191
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: tombkeeper (tombkeeper)
Assigned to: Nobody/Anonymous (nobody)
Summary: oprofied failed to open device on mpc8572 linux version 2.6.
Initial Comment:
Linux version: 2.6.31
CPU: mpc8572, e500 core
oprofile version: 0.9.7
1>oprofile --init is ok
2>oprofile --start failed, error output:
Using 2.6+ OProfile kernel interface.
Failed to open device. Possibly you have passed incorrect
parameters. Check /var/log/messages.Couldn't start oprofiled.
Check the log file "/var/lib/oprofile/samples/oprofiled.log" and kernel syslog
3>check /var/log/message, nothing there. check oprofiled.log, no such file exist.
4>oprofile -l: list all features supported by e500
I tried to use strace to find which open failed and got lots of /dev/oprofile/* opened, not sure which failed. Also walking through oprofile code is not helpful, only see opening /dev/oprofile/buffer code will output such errors.
Anyone knows how to debug this, to figure out if the issue is in kernel or application, if it is some incorrect parameter?
Thanks.
John
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=116191&aid=3528383&group_id=16191
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
</pre>SourceForge.net2012-05-20T11:43:24Build error: oprofile with CONFIG_CPUMASK_OFFSTACK=y && SMP
http://comments.gmane.org/gmane.linux.oprofile/10492
<pre>My OMAP randconfig build found this build error last night:
arch/arm/oprofile/../../../drivers/oprofile/oprofile_perf.c:28: error: variably modified 'perf_events' at file scope
This is caused because nr_cpumask_bits is a variable when we're building
for a SMP system and CONFIG_CPUMASK_OFFSTACK is set.
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
</pre>Russell King - ARM Linux2012-05-19T07:32:11Need help in cross compiling oprofile
http://comments.gmane.org/gmane.linux.oprofile/10491
<pre>Hello everyone
I need some help in cross compiling oprofile for ARM V7 architecture.
Some hints would be really helpful.
Regards
Abdul
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
</pre>Abdul Wahid Memon2012-05-18T20:18:20[PATCH] Make various minor cleanup fixes to operf
http://comments.gmane.org/gmane.linux.oprofile/10490
<pre>[PATCH] Make various minor cleanup fixes to operf
- Remove or update stale comments
- Add calls to cleanup() at error exit points
- Add pre-check to verify user has access to sample data dir
Signed-off-by: Maynard Johnson <maynardj< at >us.ibm.com>
---
libperf_events/operf_utils.cpp | 6 +----
pe_profiling/operf.cpp | 41 ++++++++++++++++++++++++++++++++++-----
2 files changed, 36 insertions(+), 11 deletions(-)
diff --git a/libperf_events/operf_utils.cpp b/libperf_events/operf_utils.cpp
index 1d18f32..4ff99c0 100644
--- a/libperf_events/operf_utils.cpp
+++ b/libperf_events/operf_utils.cpp
< at >< at > -280,7 +280,6 < at >< at > static struct operf_transient * __get_operf_trans(struct sample_data * data)
proc = it->second;
trans.cur_procinfo = proc;
} else {
-// TODO
/* This can happen for the following reasons:
* - We get a sample before getting a COMM or MMAP
* event for the process being profiled
< at >< at > -291,10 +290,7 < at >< at > static struct operf_transient * __get_operf_trans(struct sample_data * data)
* process other than the one we requested (not
* likely -- this would be a kernel bug if it did)
*
- * Need to look into caching these discarded samples and trying to
- * process them after we have a valid app name recorded. This could
- * cause a lot of thrashing about. But at the very least,
- * we need to log the lost sample.
+ * TODO: log the lost sample if !first_time_processing
*/
if ((cverb << vmisc) && !first_time_processing)
cerr << "Dropping sample -- process info unavailable" << endl;
diff --git a/pe_profiling/operf.cpp b/pe_profiling/operf.cpp
index bab5b2a..4b2c65f 100644
--- a/pe_profiling/operf.cpp
+++ b/pe_profiling/operf.cpp
< at >< at > -585,12 +585,13 < at >< at > static void complete(void)
}
if (operf_options::reset) {
int flags = FTW_DEPTH | FTW_ACTIONRETVAL;
-
+errno = 0;
if (nftw(current_sampledir.c_str(), __delete_sample_data, 32, flags) !=0 &&
errno != ENOENT) {
-cerr << "Problem encountered clearing old sample data."
- << " Possible permissions problem." << endl;
-cerr << "Try a manual removal of " << current_sampledir << endl;
+cerr << "Unable to remove old sample data at "
+ << current_sampledir << "." << endl;
+if (errno)
+cerr << strerror(errno) << endl;
cleanup();
exit(1);
}
< at >< at > -1066,8 +1067,6 < at >< at > static void process_args(int argc, char const ** argv)
_process_session_dir();
outputfile = samples_dir + "/" + DEFAULT_OPERF_OUTFILE;
-// TODO: Need to examine ocontrol to see what (if any) additional
-// event verification is needed here.
if (operf_options::evts.empty()) {
// Use default event
get_default_event();
< at >< at > -1129,14 +1128,44 < at >< at > int main(int argc, char const *argv[])
my_uid = geteuid();
if (operf_options::system_wide && my_uid != 0) {
cerr << "You must be root to do system-wide profiling." << endl;
+cleanup();
exit(1);
}
if (cpu_type == CPU_NO_GOOD) {
cerr << "Unable to ascertain cpu type. Exiting." << endl;
+cleanup();
exit(1);
}
op_nr_counters = op_get_nr_counters(cpu_type);
+
+if (my_uid != 0) {
+/* Pre-check to make sure we have permission to remove old sample data
+ * or to create new sample data in the specified sample data directory.
+ * If the user wants us to remove old data, we don't actually do it now,
+ * since the profile session may fail for some reason or the user may do ctl-c.
+ * We should exit without unnecessarily removing the old sample data as
+ * the user may expect it to still be there after an aborted run.
+ */
+string current_sampledir = samples_dir + "/current";
+string current_sampledir_testfile = current_sampledir + "/.xxxTeStFiLe";
+ofstream afile;
+errno = 0;
+afile.open(current_sampledir_testfile.c_str());
+if (!afile.is_open() && (errno != ENOENT)) {
+if (operf_options::reset)
+cerr << "Unable to remove old sample data at "
+ << current_sampledir << "." << endl;
+else
+cerr << "Unable to write to sample data directory at "
+ << current_sampledir << "." << endl;
+if (errno)
+cerr << strerror(errno) << endl;
+cerr << "Try a manual removal of " << current_sampledir << endl;
+cleanup();
+exit(1);
+}
+}
end_code_t run_result;
if ((run_result = _run())) {
if (app_started && (run_result != APP_ABNORMAL_END)) {
</pre>Maynard Johnson2012-05-17T14:12:08Should operf clear old profile data by default?
http://comments.gmane.org/gmane.linux.oprofile/10487
<pre>The new perf_events-based 'operf' tool (in the perf-events branch) currently has a "--reset" option which will clear out old profile data. If the user does not pass the "--reset" option, then new profile data that operf collects will be appended to existing profile data. I designed the interface this way to mimic the way that opcontrol-based profiling works -- i.e., the user needs to do 'opcontrol --reset' prior to profiling unless they want to append new data to old. My guess is that oprofile users very infrequently want or need to append profile data, so I am proposing that operf's interface be changed as follows:
Remove the "--reset" option and add an "--append" option.
The resulting behavior would clear out old profile data by default unless the "--append" option is passed.
Does anyone have an opinion, yea or nay, on this?
Thanks.
-Maynard
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
</pre>Maynard Johnson2012-05-16T18:32:37[PATCH] Flesh out operf JIT support so old jitdump files are removed
http://comments.gmane.org/gmane.linux.oprofile/10484
<pre>[PATCH] Flesh out operf JIT support so old jitdump files are removed
This patch makes opjitconv delete old jitdump files when
invoked by operf. Only jitdump files owned by the user who
is running operf will be deleted. This patch is necessary
because the only other way that old jitdump files are deleted
is by way of opcontrol, and operf users should not have to
use opcontrol at all.
This patch has been committed to the perf-events branch, but review
comments are welcome.
Signed-off-by: Maynard Johnson <maynardj< at >us.ibm.com>
---
opjitconv/opjitconv.c | 141 +++++++++++++++++++++++++++++++++++++++++++++++-
opjitconv/opjitconv.h | 6 ++
pe_profiling/operf.cpp | 4 +-
3 files changed, 148 insertions(+), 3 deletions(-)
diff --git a/opjitconv/opjitconv.c b/opjitconv/opjitconv.c
index 9da0c82..3583f17 100644
--- a/opjitconv/opjitconv.c
+++ b/opjitconv/opjitconv.c
< at >< at > -72,6 +72,9 < at >< at > struct jitentry ** entries_address_ascending;
int debug;
/* indicates opjitconv invoked by non-root user via operf */
int non_root;
+/* indicates we should delete jitdump files owned by the user */
+int delete_jitdumps;
+LIST_HEAD(jitdump_deletion_candidates);
/*
* Front-end processing from this point to end of the source.
< at >< at > -311,8 +314,17 < at >< at > chk_proc_id:
goto out;
}
if (!(anon_dir = find_anon_dir_match(anon_sample_dirs, proc_id))) {
-printf("Possible error: No matching anon samples for %s\n",
- dmp_pathname);
+/* With the capability of profiling with operf (as well as with
+ * the legacy oprofile daemon), users will not be using opcontrol
+ * to remove all jitdump files; instead, opjitconv will remove old
+ * jitdump files (see _cleanup_jitdumps() for details). But this cleanup
+ * strategy makes it quite likely that opjitconv will sometimes find
+ * jitdump files that are not owned by the current user or are in use
+ * by other operf users, thus, the current profile data would not have
+ * matching anon samples for such jitdump files.
+ */
+verbprintf(debug, "Informational message: No matching anon samples for %s\n",
+ dmp_pathname);
rc = OP_JIT_CONV_NO_MATCHING_ANON_SAMPLES;
goto free_res1;
}
< at >< at > -481,6 +493,42 < at >< at > static void filter_anon_samples_list(struct list_head * anon_dirs)
}
+static void _add_jitdumps_to_deletion_list(void * all_jitdumps, char const * jitdump_dir )
+{
+struct list_head * jd_fnames = (struct list_head *) all_jitdumps;
+struct list_head * pos1, *pos2;
+size_t dir_len = strlen(jitdump_dir);
+
+list_for_each_safe(pos1, pos2, jd_fnames) {
+struct pathname * dmpfile =
+list_entry(pos1, struct pathname, neighbor);
+struct stat mystat;
+char dmpfile_pathname[dir_len + 20];
+int fd;
+memset(dmpfile_pathname, '\0', dir_len + 20);
+strcpy(dmpfile_pathname, jitdump_dir);
+strcat(dmpfile_pathname,dmpfile->name);
+fd = open(dmpfile_pathname, O_RDONLY);
+if (fd < 0) {
+// Non-fatal error, so just display debug message and continue
+verbprintf(debug, "opjitconv: cannot open jitdump file %s\n",
+ dmpfile_pathname);
+continue;
+}
+if (fstat(fd, &mystat) < 0) {
+// Non-fatal error, so just display debug message and continue
+verbprintf(debug, "opjitconv: cannot fstat jitdump file");
+continue;
+}
+if (geteuid() == mystat.st_uid) {
+struct jitdump_deletion_candidate * jdc =
+xmalloc(sizeof(struct jitdump_deletion_candidate));
+jdc->name = xstrdup(dmpfile->name);
+list_add(&jdc->neighbor, &jitdump_deletion_candidates);
+}
+}
+}
+
static int op_process_jit_dumpfiles(char const * session_dir,
unsigned long long start_time, unsigned long long end_time)
{
< at >< at > -534,6 +582,9 < at >< at > static int op_process_jit_dumpfiles(char const * session_dir,
|| list_empty(&jd_fnames))
goto rm_tmp;
+if (delete_jitdumps)
+_add_jitdumps_to_deletion_list(&jd_fnames, jitdump_dir);
+
/* Get user information (i.e. UID and GID) for special user 'oprofile'.
*/
if (non_root) {
< at >< at > -611,6 +662,82 < at >< at > out:
return rc;
}
+static void _cleanup_jitdumps(void)
+{
+struct list_head * pos1, *pos2;
+char const * jitdump_dir = "/var/lib/oprofile/jitdump/";
+size_t dir_len = strlen(jitdump_dir);
+char dmpfile_pathname[dir_len + 20];
+char proc_fd_dir[PATH_MAX];
+
+if (!delete_jitdumps)
+return;
+
+/* The delete_jitdumps flag tells us to try to delete the jitdump files we found
+ * that belong to this user. Only operf should pass the --delete-jitdumps
+ * argument to opjitconv since legacy oprofile uses opcontrol to delete old
+ * jitdump files.
+ *
+ * The code below will only delete jitdump files that are not currently
+ * being used by another process.
+ */
+list_for_each_safe(pos1, pos2, &jitdump_deletion_candidates) {
+DIR * dir;
+struct dirent * dirent;
+int pid;
+size_t dmpfile_name_len;
+int do_not_delete = 0;
+struct jitdump_deletion_candidate * cand = list_entry(pos1,
+ struct jitdump_deletion_candidate,
+ neighbor);
+memset(dmpfile_pathname, '\0', dir_len + 20);
+memset(proc_fd_dir, '\0', PATH_MAX);
+
+if ((sscanf(cand->name, "%d", &pid)) != 1) {
+verbprintf(debug, "Cannot get process id from jitdump file %s\n",
+ cand->name);
+continue;
+}
+
+strcpy(dmpfile_pathname, jitdump_dir);
+strcat(dmpfile_pathname, cand->name);
+dmpfile_name_len = strlen(dmpfile_pathname);
+
+sprintf(proc_fd_dir, "/proc/%d/fd/", pid);
+if ((dir = opendir(proc_fd_dir))) {
+size_t proc_fd_dir_len = strlen(proc_fd_dir);
+while ((dirent = readdir(dir))) {
+if (dirent->d_type == DT_LNK) {
+char buf[1024];
+char fname[1024];
+memset(fname, '\0', 1024);
+strcpy(fname, proc_fd_dir);
+strncat(fname, dirent->d_name, 1023 - proc_fd_dir_len);
+if (readlink(fname, buf, 1023) > 0) {
+verbprintf(debug, "readlink found for %s\n", buf);
+if (strncmp(buf, dmpfile_pathname,
+ dmpfile_name_len) == 0) {
+do_not_delete = 1;
+break;
+}
+}
+}
+}
+}
+if (!do_not_delete)
+remove(dmpfile_pathname);
+}
+list_for_each_safe(pos1, pos2, &jitdump_deletion_candidates) {
+struct jitdump_deletion_candidate * pname = list_entry(pos1,
+ struct jitdump_deletion_candidate,
+ neighbor);
+free(pname->name);
+list_del(&pname->neighbor);
+free(pname);
+}
+
+}
+
int main(int argc, char ** argv)
{
unsigned long long start_time, end_time;
< at >< at > -630,6 +757,13 < at >< at > int main(int argc, char ** argv)
argv++;
}
+delete_jitdumps = 0;
+if (argc > 1 && strcmp(argv[1], "--delete-jitdumps") == 0) {
+delete_jitdumps = 1;
+argc--;
+argv++;
+}
+
if (argc != 4) {
printf("Usage: opjitconv [-d] <session_dir> <starttime>"
" <endtime>\n");
< at >< at > -661,6 +795,9 < at >< at > int main(int argc, char ** argv)
verbprintf(debug, "start time/end time is %llu/%llu\n",
start_time, end_time);
rc = op_process_jit_dumpfiles(session_dir, start_time, end_time);
+if (delete_jitdumps)
+_cleanup_jitdumps();
+
if (rc > OP_JIT_CONV_OK) {
verbprintf(debug, "opjitconv: Ending with rc = %d. This code"
" is usually OK, but can be useful for debugging"
diff --git a/opjitconv/opjitconv.h b/opjitconv/opjitconv.h
index 71c2c95..9562256 100644
--- a/opjitconv/opjitconv.h
+++ b/opjitconv/opjitconv.h
< at >< at > -82,6 +82,12 < at >< at > struct pathname
struct list_head neighbor;
};
+struct jitdump_deletion_candidate
+{
+char * name;
+struct list_head neighbor;
+};
+
/* jitsymbol.c */
void create_arrays(void);
int resolve_overlaps(unsigned long long start_time);
diff --git a/pe_profiling/operf.cpp b/pe_profiling/operf.cpp
index bc5310f..bab5b2a 100644
--- a/pe_profiling/operf.cpp
+++ b/pe_profiling/operf.cpp
< at >< at > -515,7 +515,7 < at >< at > static void _do_jitdump_convert()
struct timeval tv;
char end_time_str[32];
char opjitconv_path[PATH_MAX + 1];
-char * exec_args[7];
+char * exec_args[8];
struct sigaction act;
sigset_t ss;
< at >< at > -542,6 +542,7 < at >< at > static void _do_jitdump_convert()
const char * jitconv_pgm = "opjitconv";
const char * debug_option = "-d";
const char * non_root_user = "--non-root";
+const char * delete_jitdumps = "--delete-jitdumps";
gettimeofday(&tv, NULL);
end_time = tv.tv_sec;
sprintf(end_time_str, "%llu", end_time);
< at >< at > -552,6 +553,7 < at >< at > static void _do_jitdump_convert()
exec_args[arg_num++] = (char *)debug_option;
if (my_uid != 0)
exec_args[arg_num++] = (char *)non_root_user;
+exec_args[arg_num++] = (char *)delete_jitdumps;
exec_args[arg_num++] = (char *)operf_options::session_dir.c_str();
exec_args[arg_num++] = start_time_str;
exec_args[arg_num++] = end_time_str;
</pre>Maynard Johnson2012-05-16T17:39:25Perf support on ARMV7
http://comments.gmane.org/gmane.linux.oprofile/10483
<pre>Hello everyone
I need some hints on how to proceed further. I am sorry if this is not
the right place to ask this type of question.
I need to collect some performance counters using perf tool. But I am
always getting the following output.
Kernel: 2.6.35-7
Performance counter stats for 'ls':
2.624511 task-clock-msecs # 0.677 CPUs
0 context-switches # 0.000 M/sec
0 CPU-migrations # 0.000 M/sec
62 page-faults # 0.024 M/sec
<not counted> cycles
<not counted> instructions
<not counted> branches
<not counted> branch-misses
<not counted> cache-references
<not counted> cache-misses
0.003875716 seconds time elapsed
I have recompiled the kernel to enable the support for events and
counters but to no avail. Any help or tips would be beneficial.
Regards
Abdul
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
</pre>Abdul Wahid Memon2012-05-16T17:33:17spin locks sampling in 64 bit environments
http://comments.gmane.org/gmane.linux.oprofile/10482
<pre>Hi all,
We've encountered an issue with sampling spin locks using oprofile in 64 bit environments running linux 2.6.18.
Under 32 bit where we have proper results, assembly code for _spin_lock_irqsave is significantly longer than the code under 64 bit. The latter includes a jump into .text.lock.spinlock which does all the hard work.
Could it be that oprofile fails to account time spent in a text section to the correct function? It doesn't appear to be falsely accounted elsewhere.
Any insights will be appreciated.
Thanks,
Jonathan
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
oprofile-list mailing list
oprofile-list< at >lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oprofile-list
</pre>Jonathan Kukin2012-05-15T17:03:07[PATCH] Fix patch committed on Feb 29, 2012 so that samples fromPID 0 are not discarded.
http://comments.gmane.org/gmane.linux.oprofile/10479
<pre>[PATCH] Fix patch committed on Feb 29, 2012 so that samples from PID 0 are not discarded.
Without this patch, we were not recording samples from process 0. Additionally, we were
incorrectly incrementing the new OPD_NO_APP_KERNEL_SAMPLE stat for such discarded samples.
It's a straightforward, obvious fix, so I committed it already. But review comments are
still welcome.
Signed-off-by: Maynard Johnson <maynardj< at >us.ibm.com>
---
daemon/opd_sfile.c | 12 +++++++++---
1 files changed, 9 insertions(+), 3 deletions(-)
diff --git a/daemon/opd_sfile.c b/daemon/opd_sfile.c
index 170abbf..7f4a01c 100644
--- a/daemon/opd_sfile.c
+++ b/daemon/opd_sfile.c
< at >< at > -294,9 +294,9 < at >< at > struct sfile * sfile_find(struct transient const * trans)
opd_stats[OPD_LOST_KERNEL]++;
return NULL;
}
-// We *know* that PID 1 and 2 are pure kernel context tasks, so
+// We *know* that PID 0, 1, and 2 are pure kernel context tasks, so
// we always want to keep these samples.
-if ((trans->tgid == 1) || (trans->tgid == 2))
+if ((trans->tgid == 0) || (trans->tgid == 1) || (trans->tgid == 2))
goto find_sfile;
// Decide whether or not this kernel sample should be discarded.
< at >< at > -312,7 +312,9 < at >< at > struct sfile * sfile_find(struct transient const * trans)
found = 1;
if (kcmd->has_cmdline) {
verbprintf(vsamples,
- "Dropping user context kernel samples due to no app cookie available.\n");
+ "Dropping user context kernel sample 0x%llx "
+ "for process %u due to no app cookie available.\n",
+ (unsigned long long)trans->pc, trans->tgid);
opd_stats[OPD_NO_APP_KERNEL_SAMPLE]++;
return NULL;
}
< at >< at > -329,6 +331,10 < at >< at > struct sfile * sfile_find(struct transient const * trans)
if(fd==-1) {
// Most likely due to process ending, so we'll assume it used to have a cmdline
kcmd->has_cmdline = 1;
+verbprintf(vsamples,
+ "Open of /proc/%u/cmdline failed, so dropping "
+ "kernel sameple 0x%llx\n",
+ trans->tgid, (unsigned long long)trans->pc);
opd_stats[OPD_NO_APP_KERNEL_SAMPLE]++;
dropped = 1;
} else {
</pre>Maynard Johnson2012-05-14T16:40:09Cannot find performance counters on (Xen) Virtual CPU
http://comments.gmane.org/gmane.linux.oprofile/10465
<pre>Hi guys,
When I tried to start Oprofile on Xen virtual machines, I encountered some problem.
Physical CPU: Intel E5620.
Xen : version 4.1.1
XenOprofile : version 0.9.5
Oprofile installed on VM: version 0.9.7
Problem:
The opcontrol cannot start on VM (dom1), while it works well on dom0 .
Scene:
Run cmd on dom0:
#opcontrol --start-daemon --active-domains=0,1 --event=LLC_MISSES:100000:0:1:1 --xen=/boot/xen-syms-4.1.1 --vmlinux=/usr/src/kernels/vmlinux
It's OK. Output:
Using 2.6+ OProfile kernel interface.
Reading module info.
Using log file /var/lib/oprofile/samples/oprofiled.log
Daemon started
But When I run cmd on dom1:
#opcontrol --reset
#opcontrol --start --vmlinux=/usr/src/linux-source-2.6.32/vmlinux --xen=/boot/xen-syms-4.1.1
Error. Output:
Not enough hardware counters. Need 1 counters but only has 0.
and, when i run:
#ls /dev/oprofile/
Output: backtrace_depth buffer buffer_size buffer_watershed cpu_buffer_size cpu_type dump enable pointer_size stats
i.e. there is no performance counter found in VM (dom1)
P.S. What I have tried to solve the problem (but do not work...)
1st: #cat /proc/cpuinfo
Get cpu family : 6
model : 44
2nd: add cpu model "0x2c" to recompile the linux kernel on VM.
But the problem is still there.
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
</pre>Tao Sun2012-05-11T00:39:21Need Help
http://comments.gmane.org/gmane.linux.oprofile/10441
<pre>Dear There,
I am trying to install oprofile on LINUX Octeon 2.6.32.27-cavium-octeon
but when I run the Command ./configure --with-linux, i get two problems
1. no kernel support for oprofile
2. no suitably configured kernel include tree found
need help to solve this problem
</pre>Farrukh Mahmood2012-05-08T13:10:22[PATCH] Fix regression to operf system-wide profiling caused byprevious patch
http://comments.gmane.org/gmane.linux.oprofile/10435
<pre>[PATCH] Fix regression to operf system-wide profiling caused by previous patch
Signed-off-by: Maynard Johnson <maynardj< at >us.ibm.com>
---
libperf_events/operf_process_info.h | 39 ++++++++++++++++++-----------------
libperf_events/operf_utils.cpp | 2 +-
2 files changed, 21 insertions(+), 20 deletions(-)
diff --git a/libperf_events/operf_process_info.h b/libperf_events/operf_process_info.h
index 1d9a791..ab4ff94 100644
--- a/libperf_events/operf_process_info.h
+++ b/libperf_events/operf_process_info.h
< at >< at > -51,21 +51,13 < at >< at > struct operf_mmap {
class operf_process_info {
public:
operf_process_info(pid_t tgid, const char * appname, bool app_arg_is_fullname, bool is_valid);
-bool is_valid(void) { return (valid && appname_valid()); }
+bool is_valid(void) { return (valid); }
void process_new_mapping(struct operf_mmap * mapping);
void process_deferred_mappings(std::string app_shortname);
std::string get_app_name(void) { return _appname; }
void add_deferred_mapping(struct operf_mmap * mapping)
{ deferred_mmappings[mapping->start_addr] = mapping; }
const struct operf_mmap * find_mapping_for_sample(u64 sample_addr);
-private:
-typedef enum {
-NOT_FULLNAME,
-MAYBE_FULLNAME,
-YES_FULLNAME
-} op_fullname_t;
-pid_t pid;
-std::string _appname;
/* The valid bit is set when a COMM event has been received for the process
* represented by this object. But since the COMM event only gives a shortname
< at >< at > -75,19 +67,12 < at >< at > private:
* result could be from:
* (appname_is_fullname == MAYBE_FULLNAME) &&(num_app_chars_matched > 0)
* But this is the best guess we can make.
- * So is_valid() has to consider both the 'valid' field and the result of
- * appname_valid().
*/
-bool valid;
-op_fullname_t appname_is_fullname;
-std::string app_basename;
-int num_app_chars_matched;
-std::map<u64, struct operf_mmap *> mmappings;
-std::map<u64, struct operf_mmap *> deferred_mmappings;
-int get_num_matching_chars(std::string mapped_filename, std::string & basename);
-bool appname_valid(void)
+bool is_appname_valid(void)
{
bool result;
+if (!valid)
+return false;
if (appname_is_fullname == YES_FULLNAME)
result = true;
else if ((appname_is_fullname == MAYBE_FULLNAME) &&
< at >< at > -98,6 +83,22 < at >< at > private:
return result;
}
+private:
+typedef enum {
+NOT_FULLNAME,
+MAYBE_FULLNAME,
+YES_FULLNAME
+} op_fullname_t;
+pid_t pid;
+std::string _appname;
+bool valid;
+op_fullname_t appname_is_fullname;
+std::string app_basename;
+int num_app_chars_matched;
+std::map<u64, struct operf_mmap *> mmappings;
+std::map<u64, struct operf_mmap *> deferred_mmappings;
+int get_num_matching_chars(std::string mapped_filename, std::string & basename);
+
};
extern std::map<pid_t, operf_process_info *> process_map;
diff --git a/libperf_events/operf_utils.cpp b/libperf_events/operf_utils.cpp
index 0da0359..1d18f32 100644
--- a/libperf_events/operf_utils.cpp
+++ b/libperf_events/operf_utils.cpp
< at >< at > -276,7 +276,7 < at >< at > static struct operf_transient * __get_operf_trans(struct sample_data * data)
} else {
// Find operf_process info for data.tgid.
std::map<pid_t, operf_process_info *>::const_iterator it = process_map.find(data->pid);
-if (it != process_map.end() && (it->second->is_valid())) {
+if (it != process_map.end() && (it->second->is_appname_valid())) {
proc = it->second;
trans.cur_procinfo = proc;
} else {
</pre>Maynard Johnson2012-05-03T19:14:48[PATCH] Make new --separate-thread option for operf
http://comments.gmane.org/gmane.linux.oprofile/10434
<pre>[PATCH] Make new --separate-thread option for operf
This patch reverses the earlier decision to do "separate=thread" type
of functionality by default. So now there is a command line option
to operf called "--separate-thread". I decided to make this an option
for two reasons: 1) operf run in system-wide mode required the user to
either pass a tgid spec or to use "--merge=tgid" or the resulting report
was unreadable; 2) typically, the Java JIT compiler will spawn threads
to execute JITed code; so when using operf to profile a single Java app,
the resulting report is usually unreadable without using a tid spec or
"--merge=tid".
Signed-off-by: Maynard Johnson <maynardj< at >us.ibm.com>
---
doc/operf.1.in | 32 ++++++++++++++++++++++----------
libperf_events/operf_mangling.cpp | 8 +++++---
libperf_events/operf_utils.h | 1 +
pe_profiling/operf.cpp | 10 +++-------
4 files changed, 31 insertions(+), 20 deletions(-)
diff --git a/doc/operf.1.in b/doc/operf.1.in
index c279323..00b6b2c 100644
--- a/doc/operf.1.in
+++ b/doc/operf.1.in
< at >< at > -12,10 +12,9 < at >< at > operf \- Performance profiler tool for Linux
[ --system-wide | --pid <pid> | [ command [ args ] ] ]
.SH DESCRIPTION
-Operf is a Linux tool that can be used in place of opcontrol for profiling. Operf
-uses Linux Performance Events Subsystem, and hence, does not need the opcontrol daemon to be running
-when running operf. More than that, opcontrol shouldn't be running when running
-operf.
+Operf is an OProfile tool that can be used in place of opcontrol for profiling. Operf
+uses the Linux Performance Events Subsystem, and hence, does not require the use of
+the opcontrol daemon -- in fact, operf and opcontrol usage are mutually exclusive.
.P
By default, operf stores profiling data in <current_dir>/oprofile_data. You can change
this by way of the
< at >< at > -72,11 +71,6 < at >< at > directory should be /root or a subdirectory of /root to avoid
storing sample data files in locations accessible by regular users.
.br
.TP
-.BI "--verbose / -V [level]"
-A comma-separated list of debugging control values, used to increase the verbosity of the output.
-Valid values are: debug, perf_events, misc, sfile, arcs, or the special value, 'all'.
-.br
-.TP
.BI "--callgraph / -g"
This option enables the callgraph to be saved during profiling. NOTE: The
full callchain is recorded, so there is no depth limit.
< at >< at > -95,7 +89,20 < at >< at > factor of <buffersize>.
This option is for passing a comma-separated list of event specifications
for profiling. Each event spec is of the form:
.br
-name:count[:unitmask[:kernel[:user]]]
+.I " name:count[:unitmask[:kernel[:user]]]"
+.br
+When no event specification is given, the default event for the running
+processor type will be used for profiling.
+.br
+.TP
+.BI "--separate-thread / -t"
+This option categorizes samples by thread group ID (tgid) and thread ID (tid).
+The '--separate-thread' option is useful for seeing per-thread samples in
+multi-threaded applications. When used in conjuction with the '--system-wide'
+option, the '--separate-thread' option is also useful for seeing per-process
+(i.e., per-thread group) samples for the case where multiple processes are
+executing the same program during a profiling run.
+.br
.TP
.BI "--separate-cpu / -c"
This option categorizes samples by cpu.
< at >< at > -108,6 +115,11 < at >< at > the data is saved in the
directory on the current path.
.br
.TP
+.BI "--verbose / -V [level]"
+A comma-separated list of debugging control values, used to increase the verbosity of the output.
+Valid values are: debug, perf_events, misc, sfile, arcs, or the special value, 'all'.
+.br
+.TP
.BI "--version / -v"
Show operf version.
.br
diff --git a/libperf_events/operf_mangling.cpp b/libperf_events/operf_mangling.cpp
index f39c64c..5b31e29 100644
--- a/libperf_events/operf_mangling.cpp
+++ b/libperf_events/operf_mangling.cpp
< at >< at > -68,9 +68,11 < at >< at > mangle_filename(struct operf_sfile * last, struct operf_sfile const * sf, int co
values.image_name = sf->image_name;
}
values.dep_name = sf->app_filename;
-values.flags |= MANGLE_TGID | MANGLE_TID;
-values.tid = sf->tid;
-values.tgid = sf->tgid;
+if (operf_options::separate_thread) {
+values.flags |= MANGLE_TGID | MANGLE_TID;
+values.tid = sf->tid;
+values.tgid = sf->tgid;
+}
if (operf_options::separate_cpu) {
values.flags |= MANGLE_CPU;
diff --git a/libperf_events/operf_utils.h b/libperf_events/operf_utils.h
index 8a6e457..060ee2b 100644
--- a/libperf_events/operf_utils.h
+++ b/libperf_events/operf_utils.h
< at >< at > -31,6 +31,7 < at >< at > extern int pid;
extern int mmap_pages_mult;
extern std::string session_dir;
extern bool separate_cpu;
+extern bool separate_thread;
}
extern bool no_vmlinux;
diff --git a/pe_profiling/operf.cpp b/pe_profiling/operf.cpp
index cff374f..bc5310f 100644
--- a/pe_profiling/operf.cpp
+++ b/pe_profiling/operf.cpp
< at >< at > -91,6 +91,7 < at >< at > int mmap_pages_mult;
string session_dir;
string vmlinux;
bool separate_cpu;
+bool separate_thread;
vector<string> evts;
}
< at >< at > -120,6 +121,8 < at >< at > popt::option options_array[] = {
"events"),
popt::option(operf_options::separate_cpu, "separate-cpu", 'c',
"Categorize samples by cpu"),
+popt::option(operf_options::separate_thread, "separate-thread", 't',
+ "Categorize samples by thread group and thread ID"),
};
}
< at >< at > -617,13 +620,6 < at >< at > static void complete(void)
cerr << endl << "Use '--session-dir=" << operf_options::session_dir << "'" << endl
<< "with opreport and other post-processing tools to view your profile data."
<< endl;
-if (operf_options::system_wide)
-cerr << "\nNOTE: The system-wide profile you requested was collected "
-"on a per-process basis." << endl
-<< "Adding '--merge=tgid' when using post-processing tools will make the output"
-<< endl << "more readable." << endl;
-
-cerr << "^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^" << endl;
} catch (runtime_error e) {
cerr << "Caught exception from operf_read::convertPerfData" << endl;
cerr << e.what() << endl;
</pre>Maynard Johnson2012-05-03T19:52:56Search EngineSearch the mailing list at Gmanequery
http://search.gmane.org/?group=$group=gmane.linux.oprofile