gmane.comp.jakarta.log4j.user

Re: Logfiles containing 'java' as methodname for %M instead of real method name

Curt Arnold — 2008-09-07T20:15:13

On Sep 6, 2008, at 9:11 AM, Thomas Wiedmann wrote: Prior to JDK 1.4, there was not a mechanism to determine the stack trace without parsing the output of Throwable.printStackTrace(). Different JVM's generate slightly different output and may potentially confuse the stack trace parser. The current SVN HEAD will detect when running on a JDK 1.4 and later and use Throwable.getStackTrace() and should be immune to formatting issues. Previous versions of log4j have had some incremental improvement in stack trace parsing (JRockit and GCJ in particular had issues). If you are using a much older version updating to log4j 1.2.15 may help. log4j 1.2.16 with the conditional use of JDK 1.4 methods should be released shortly.

Logfiles containing 'java' as methodname for %M instead of real method name

Thomas Wiedmann — 2008-09-06T14:11:40

Hello, in the ...layout.ConversionPattern of a DailyRollingFileAppender the %M parameter is defined to output the name of the method where the logging was triggered. In some logfiles instead of the real method name the entry 'java' is found. What may be the reason for this behaviour and how to achieve the output of the name of the triggering method? Thomas Wiedmann

Re: network logging performance

Paul Smith — 2008-09-05T02:44:26

[oops, I originally sent this reply direct to Johannes. Copying again the list] On 04/09/2008, at 4:19 PM, Johannes Gutleber wrote: fantastic. awesome to hear about this sort of environment. Really appreciate you taking the time to respond in depth here. Are you able to articulate where you thought the bottleneck within the Java process was ? What JRE were you using? Was it GC overhead? Was it log4j itself, or the custom built collector that was the problem? Which version of Chainsaw did you use? Was it the one inbuilt into log4j, or the newer Chainsaw v2? The latter uses a cyclic buffer so as not to consume too much memory. I do agree though that even Chainsaw v2 is far from perfect.. :) My vision, ridiculous as this may sound, is that log4j should be able to support an environment in the new cloud environments. Imagine the Hadoop clusters that are out there and making sense of wtf happened during the dissemination of the MapReduce portions, with thousands of host n

RE: network logging performance

Bruno Melloni — 2008-09-04T13:23:04

**Theoretically**, a logging event should be a 'fire and forget' activity from the perspective of the application. That means that you should be able to submit the logging event to the 'logging system' without blocking, and then the 'logging system' should be able to deliver those logging events to the final destination on its own thread(s) and at its own speed. As long as the average volume of logging does not exceed the capacity of the actual logging mechanism (i.e.: over the network) then there ought to be no impact whatsoever to the application (except perhaps for the cpu overhead of the physical logging thread(s)). I don't know log4j well enough to say whether it is already doing this isolation of 'log event submission' vs 'log event processing'. If it doesn't, you should be able to have your application submit all logging requests through a class written by you that puts all requests into a memory cache, ensuring no blocking. Then have one or more separate threads handling the 'log event processin

CLOB datatype column usage with JDBCPlus jdbc appender

Vaibhav Kumar — 2008-09-04T09:56:58

Hi, I am using JDBCPlus (http://www.dankomannhaupt.de/projects/index.html) as a jdbc appender for logging into database. For a requirement, I had to change the datatype of a column (which stores the message string to be logged) from varchar2(4000) to CLOB. But as soon I do this, the logging in database stops and the stack trace got is : log4j:ERROR JDBCAppender::flush_buffer(), : java.sql.SQLException: Internal Error: Unable to construct a Datum from the specified input at oracle.jdbc.dbaccess.DBError.throwSqlException(DBError.java:168) at oracle.jdbc.dbaccess.DBError.throwSqlException(DBError.java:210) at oracle.jdbc.dbaccess.DBError.check_error(DBError.java:829) at oracle.sql.SQLUtil.makeDatum(SQLUtil.java:645) at oracle.sql.SQLUtil.makeOracleDatum(SQLUtil.java:946) at oracle.jdbc.driver.UpdatableResultSet.updateObject(UpdatableResultSet.java:1568) at oracle.jdbc.driver.OracleResultSet.updateObject(OracleResultSet.java:2787) at org.apache.log4j.jdbcplus.JDBCLogger.append(JD

RE: network logging performance

Michael Erskine — 2008-09-04T09:14:13

For convenience I routinely configure SocketHubAppender instances for all my services and typically set a threshold of WARN to reduce traffic to important info. Where performance is deemed more important than the timely delivery of the information I have custom asynchronous appenders with limited BlockingDeque queues and daemon threads to pump out data at their leisure. Where I have multiple services logging across the network to a central place I tend to use a custom JDBC asynchronous appender. In other scenarios I have networked systems that require certain log messages as part of a larger communications protocol. Here I have custom appenders that catch LoggingEvent messages, translate them and drop them in the outgoing BlockingDeque for the target system (I always work with queues on complex systems to encourage loose coupling). I don't tend to pass Java objects over the wire: call me a traditionalist but when I write my protocols they tend to be human-readable 7-bit ASCII where you can telnet to a s

network logging performance

Paul Smith — 2008-09-04T01:31:23

If anyone out there is using "Logging over the network" of any form (socket, JMS, Multicast, syslog appenders etc), this topic is for you. I'm wondering whether people could comment on their experience setting up high performance logging of application data over the network. The cost of shipping an event over the wire compared with writing it to a local file is obviously higher, and so one can notice user-visible impact if the logging volume is high when using network-based logging (unless one wraps it in AsyncAppenders). Just curious to hear peoples experiences, strategies, and thoughts. Perhaps people can relate to the flow rate they've manage to achieve under different configurations. The driver to my question is that for my Apache Lab project (Pinpoint) I'm building a central logging repository server to allow data mining of the generated logging events. In a high performance site one can have many logging events shipped from multiple threads in, say, a web- app, being shipp

RE: New Appender: PatternFileAppender

Bruno Melloni — 2008-09-03T19:12:04

Seems that several of us have been looking at similar things. I looked at the MultiFile appender but it wasn't quite flexible enough, so I created another appender similar to this (DynamicTimestampedFileAppender) and was debating whether to submit it. It is complete but has only been in use for under a week. Please let me know if I should submit it, or perhaps the three appenders ought to be compared and eventually create a hybrid appender that replaces them all. Key features: - Designed as a full lifecycle file appender. - Instead of sequence numbers uses timestamps of file creation (much easier to find entries). Rollovers are really nothing more than closing a file and creating a new one. - Zips and deletes old files. Zips and deletes happen at night and server start (minimizes load when server is running). - Multiple logging file sets are dynamically generated. - Has built-in variables that can be used in defining the file names. - Additional variables can be defined by parsing the message thro

Re: Future of JULI<->Log4j bridging

Paul Smith — 2008-09-03T04:57:55

wow! cool! this is great news to hear. Now we have 2 users! :) If you're interested in trialing things one day, look out for Pinpoint (currently an Apache Lab). It's for log data mining. centralised log repository with full text indexing and reporting. I'm right in the middle of a load test for it as we speak (soaking events from 2 hosts, at 100 events/second, this is a Mercury load test of our actual application running realistic production loads). Unfortunately there's no real UI for actually doing the searching as yet, but the indexing side is working very well. cheers, Paul On 03/09/2008, at 2:20 PM, Brett Randall wrote: Paul Smith Core Engineering Manager Aconex The easy way to save time and money on your project 696 Bourke Street, Melbourne, VIC 3000, Australia Tel: +61 3 9240 0200 Fax: +61 3 9240 0299 Email: psmith< at >aconex.com www.aconex.com This email and any attachments are intended solely for the addressee. The contents may be privileged, confidential and/or subjec

Re: Future of JULI<->Log4j bridging

Brett Randall — 2008-09-03T04:20:44

Thanks - all good and working for now. Bridge version apache-jul-log4j-bridge-1.0.0-20071030.02281 appears to work well. Using Spring Framework, I can actually bootstrap the bridge during the Spring context load using this: org.apache.logging.julbridge.JULLog4jBridge.assimilate ... which I think is nice. No reference or dependency on the jul-log4j-bridge in my code, deploy-time replacement of JUL backend with log4j. Then I can even plug-in Chainsaw: (log4j.xml) logging.apache.org> Subject: Re: DEBUG messages appearing on console? On Sep 2, 2008, at 2:00 PM, Patrick.Grimard< at >xtl.com wrote: The filter suggestion is a bit overkill for what you need. What is happening is that you are assigning a DEBUG threshold to all logging requests to loggers starting with "com.dsn.loadlink", all other loggers have a threshold of WARN. You are also attaching a ConsoleAppender to the root logger and a RollingFileAppender to "com.dsn.loadlink", but that is an independent action from setting the thresholds. So if you make a logging request on "com.dsn.loadlink" or a child logger, it is evaluated against the nearest specified threshold and if that is satisfied, then it is processed by all appenders att

Re: New Appender: PatternFileAppender

David Britton — 2008-09-02T21:32:59

Hi Curt -- Glad I asked! It looks to be nearly identical in base functionality, but of course has a much more developed interface, and has many more options to it. It looks like it has two main "policies", a Pattern policy and a Size policy. The "Size" policy looks like it provides duplicate functionality that the RollingFileAppender provides. It increments an index appended to the file after they get to a certain size. I may be over simplifying here -- but I think that RollingFileAppender provides this and more. The "Pattern" policy is what my appender was supposed to solve, and I think is what is "missing" from log4j right now. Just from inspection, it appears as though it would be slightly more complex than my PatternFileAppender, but has more configuration options, and of course is 1.2 compatible. :) Is there a reason this has not made it into the official distribution?

Re: New Appender: PatternFileAppender

Curt Arnold — 2008-09-02T21:04:30

On Sep 2, 2008, at 3:38 PM, David Britton wrote: Could you compare it with the MultiFileAppender that has been languishing in the sandbox for a while? Search the archives for MultiFileAppender. Source code at http://svn.apache.org/repos/asf/logging/sandbox/log4j/multifile .

Re: DEBUG messages appearing on console?

Curt Arnold — 2008-09-02T20:47:20

On Sep 2, 2008, at 2:00 PM, Patrick.Grimard< at >xtl.com wrote: The filter suggestion is a bit overkill for what you need. What is happening is that you are assigning a DEBUG threshold to all logging requests to loggers starting with "com.dsn.loadlink", all other loggers have a threshold of WARN. You are also attaching a ConsoleAppender to the root logger and a RollingFileAppender to "com.dsn.loadlink", but that is an independent action from setting the thresholds. So if you make a logging request on "com.dsn.loadlink" or a child logger, it is evaluated against the nearest specified threshold and if that is satisfied, then it is processed by all appenders attached in the hierarchy (unless additivity is set to false which will prevent appenders attached to parent loggers from seeing the event). You can set a threshold on the appender which is independent of the threshold on the logger. Unless you have a specific need to separate the appenders, I would attach them both to root, but s

New Appender: PatternFileAppender

David Britton — 2008-09-02T20:38:26

Hello -- I'm not sure if this is the correct way to do this, but my colleague and I have developed a new appender called a PatternFileAppender. I did not see anything else like this in the log4j distribution, but it helped us solve our particular problem, and I think it's a good general purpose appender. If someone knows of another appender like this, I would love to hear about it. I went to make an official patch for log4j 1.2, but I found that the code needed to be compatible with 1.2 source compatibility level. Since the code uses generics in it's use of the LinkedHashMap data structure, it needs 1.5 or later to compile correctly. Anyway, I've attached the file, if there is an "official" way to do this, please point me in the right direction. Here is the high-level details of the appender: * Can process a filename as a pattern, with all substitutions available. (e.g., MDC, date, variable, level, class, ...) * Interesting use cases we are using it for: - Separate log file per "thread" using

Re: DEBUG messages appearing on console?

David Britton — 2008-09-02T19:31:47

Please see the following answer I recently gave: http://mail-archives.apache.org/mod_mbox/logging-log4j-user/200808.mbox/%3C20080825151619.GA13754< at >dune.americas.cpqcorp.net%3E