gmane.network.tin.devel

[PATCH] do not decode supersedes header

Dennis Preiser — 2013-04-29T17:19:16

Hello,

tin displays a decoded version of the Supersedes:-header of this
article (Supersedes in news_headers_to_display or after
PageToggleAllHeaders '*'):

| From: Guido Hennecke <nospam-2012-remove-this< at >usenet.debian.dyndns.org>
| Newsgroups: de.sci.physik,de.comm.software.newsreader
| Date: Fri, 26 Apr 2013 23:36:05 +0200
| Subject: Re: Das Garagen-Paradox der ART
| Supersedes: =?UTF-8?Q?=3Cslrnknlsl6=2Epb4=2Enospam=2D2012=2Dremove=2Dthi?=
|  =?UTF-8?Q?s=40msgid=2Eusenet=2Edebian=2Edyndns=2Eorg=3E?=
| Message-ID: <slrnknlsq5.pb4.nospam-2012-remove-this< at >msgid.usenet.debian.dyndns.o
rg>

This header is broken (encoded, folded), but displaying a decoded
version seems not the best choice to me. With the attached patch tin no
longer decodes the Supersede:-headers before displaying.

Dennis
diff -urp tin-2.1.2_r4/src/cook.c tin-2.1.2_r5/src/cook.c
--- tin-2.1.2_r4/src/cook.c2012-12-24 11:22:00.000000000 +0100
+++ tin-2.1.2_r5/src/cook.c2013-04-29 17:50:02.000000000 +0200
< at >< at > -872,7 +872,7 < at >< at > cook_article(

Re: tin problem: Error: From: line missing.

Urs Janßen — 2013-01-04T16:34:20

Looks like I traced it down:

if the locale charset is the same as the network charset (in your case both
were KOI8-R) tin tries to convert the string in the local charset to the
network charset (waste of cpu cycles, but sould work!). But FreeBSDs
iconv(3) seems to have a problem with that, the mapage states:

[...]
| After calling iconv(), the values pointed to by src,
| srcleft, dst, and dstleft are updated as follows:
[...]
|     *dst      Pointer to the byte just after the last character stored.
[...]

but this is not the case, right befor the conversion
(misc.c:buffer_to_network()):

(gdb) print inbuf
$2 = 0x80163d200 "From: Urs Janssen <urs< at >tin.org>"
(gdb) print &outbuf
$3 = (char **) 0x7fffffffa5e0
(gdb) print outbuf
$4 = 0x80144d6c0 "G"

right after the conversin:
(gdb) print *&outbuf
$11 = 0x80144d6c0 "From: Urs Janssen <urs< at >tin.org>l"
(gdb) print **&outbuf
$17 = 70 'F'

but it should point right behind the '>' to the tailing gargabe ('l').

(gdb) next
2560                    **&outbuf = '\0';

now

[ANNOUNCE] tin 2.1.2 released

Urs Janßen — 2012-12-24T11:05:56

tin 2.1.2 includes some minor bugfixes

<ftp://ftp.tin.org/pub/news/clients/tin/v2.1/CHANGES>

<ftp://ftp.tin.org/pub/news/clients/tin/v2.1/tin-2.1.2.tar.gz>
<ftp://ftp.tin.org/pub/news/clients/tin/v2.1/tin-2.1.2.tar.bz2>
<ftp://ftp.tin.org/pub/news/clients/tin/v2.1/tin-2.1.2.tar.xz>

[tin 2.1.1] convert username/password to UTF-8 for SASL AUTH PLAIN

Urs Janßen — 2012-11-06T14:46:58

follow RFC 4616 (untested!)

=== modified file 'src/auth.c'
--- src/auth.c2011-12-24 15:47:48 +0000
+++ src/auth.c2012-11-06 14:35:26 +0000
< at >< at > -427,15 +427,54 < at >< at >
 {
 char line[PATH_LEN];
 char *foo;
+char *utf8user;
+char *utf8pass;
 int ret;
+#ifdef CHARSET_CONVERSION
+int i, c = 0;
+#endif /* CHARSET_CONVERSION */
+
+utf8user = strdup(authuser);
+utf8pass = strdup(authpass);
+#ifdef CHARSET_CONVERSION
+if (!IS_LOCAL_CHARSET("UTF-8")) {
+for (i = 0; txt_mime_charsets[i] != NULL; i++) {
+if (!strcasecmp("UTF-8", txt_mime_charsets[i])) {
+c = i;
+break;
+}
+}
+if (c == i) { /* should never fail */
+if (!buffer_to_network(utf8user, c)) {
+free(utf8user);
+free(utf8pass);
+utf8user = strdup(authuser);
+utf8pass = strdup(authpass);
+} else {
+if (!buffer_to_network(utf8pass, c)) {
+free(utf8user);
+free(utf8pass);
+utf8user = strdup(authuser);
+utf8pass = strdup(authpass);
+}
+}
+}
+}
+#endif /* CHARSET_CONVERSION */

[tin 2.1.1] bug in check_moderated()

Urs Janßen — 2012-07-12T21:56:06

(re)posting to a group not available on the server wasn't a good idea
...

--- post.c2012-05-30 23:03:31.000000000 +0200
+++ post.c2012-07-12 23:52:31.061621983 +0200
< at >< at > -2053,6 +2053,7 < at >< at >
 const char *failmsg)
 {
 char *groupname;
+char *ogroupn;
 char newsgroups[HEADER_LEN];
 struct t_group *group;
 struct t_group *first_group = NULL;
< at >< at > -2061,7 +2062,7 < at >< at >
 /* Take copy - strtok() modifies its args */
 STRCPY(newsgroups, groups);
 
-groupname = strtok(newsgroups, ",");
+ogroupn = groupname = strtok(newsgroups, ",");
 
 do {
 vnum++; /* number of newsgroups */
< at >< at > -2110,7 +2111,7 < at >< at >
 if (vnum > bnum)
 return first_group;
 else {
-error_message(2, _(txt_not_in_active_file), groupname);
+error_message(2, _(txt_not_in_active_file), ogroupn);
 return NULL;
 }
 }

[ANNOUNCE] tin 2.1.1 released

Urs Janßen — 2012-06-23T10:15:25

tin 2.1.1 includes some mainly minor bugfixes and a new translation
(zh_TW).

<ftp://ftp.tin.org/pub/news/clients/tin/v2.1/CHANGES>

<ftp://ftp.tin.org/pub/news/clients/tin/v2.1/tin-2.1.1.tar.gz>
<ftp://ftp.tin.org/pub/news/clients/tin/v2.1/tin-2.1.1.tar.bz2>
<ftp://ftp.tin.org/pub/news/clients/tin/v2.1/tin-2.1.1.tar.lzma>

urs

Re: Issue with tin and pam_mktemp

Urs Janßen — 2012-06-20T17:27:00

this patch (against tin-2.1.0) should fix your issue (untested!)

=== modified file 'include/tin.h'
--- include/tin.h2012-06-07 14:29:14 +0000
+++ include/tin.h2012-06-20 17:22:12 +0000
< at >< at > -581,11 +581,11 < at >< at >
 #ifdef HAVE_LONG_FILE_NAMES
 #define PATH_PART"part"
 #define PATH_PATCH"patch"
-#define INDEX_LOCK"%stin.%s.LCK"
+#define INDEX_LOCK"tin.%s.LCK"
 #else
 #define PATH_PART""
 #define PATH_PATCH"p"
-#define INDEX_LOCK"%s%s.LCK"
+#define INDEX_LOCK"%s.LCK"
 #endif /* HAVE_LONG_FILE_NAMES */
 
 /*

=== modified file 'src/init.c'
--- src/init.c2012-05-17 12:34:01 +0000
+++ src/init.c2012-06-20 17:21:50 +0000
< at >< at > -636,6 +636,7 < at >< at >
 FILE *fp;
 char *ptr;
 const char *cptr;
+char tmp[PATH_LEN];
 struct stat sb;
 struct passwd *myentry;
 
< at >< at > -882,7 +883,8 < at >< at >
 joinpath(postponed_articles_file, sizeof(postponed_articles_file), rcdir, POSTPONED_FILE);
 joinpath(save_active_file, sizeof(save_active_file), rcdir, ACTIVE_SAVE_FILE);
 
-snprintf(lock_file, sizeof(lock_file), INDEX_LOCK, TMP

[tin 2.1.1] new snaphsots

Urs Janßen — 2012-06-07T14:27:30

I've made new snapshots of the upcomming tin 2.1.1 (unstable)
release including just a few minor cleanups and the fix for
chached overview data when mime is not used for 8bit chars

<ftp://ftp.tin.org/pub/news/clients/tin/unstable/snapshots/>

[tin 2.1.1] snaphsots

Urs Janßen — 2012-03-05T09:42:18

I've made new snapshots of the upcomming tin 2.1.1 (unstable) release
fixing a bug with toggleable configurations and using the 'M'enu.
Configure now looks for libunistring if libicuuc is not found to use
for unicode normalization (if one of both is found the default
normalization when none is set in tinrc is now NFC as suggested by
RFC 6532 istead of NFKC which is only used if we only have libidn or
the user explecitely requests it).
xface.c and heapsort.c are now compiled conditionally.

<ftp://ftp.tin.org/pub/news/clients/tin/unstable/snapshots/>

urs

Re: [tin 2.1.1] snaphsots

Dennis Preiser — 2012-02-21T17:10:39

I think this is not group specific. We can undo the changes which would
be necessary only for group specific attributes (read_attributes_file(),
attribute state).

Dennis

diff -urp tin-2.1.1/include/tin.h tin-2.1.1_r1/include/tin.h
--- tin-2.1.1/include/tin.h2012-02-21 16:58:50.000000000 +0100
+++ tin-2.1.1_r1/include/tin.h2012-02-21 17:20:37.000000000 +0100
< at >< at > -1672,9 +1672,6 < at >< at > struct t_attribute_state {
 unsigned signature_repost:1;
 unsigned sort_article_type:1;
 unsigned sort_threads_type:1;
-#ifdef USE_HEAPSORT
-unsigned sort_function:1;
-#endif
 unsigned start_editor_offset:1;
 unsigned tex2iso_conv:1;
 unsigned thread_articles:1;
diff -urp tin-2.1.1/src/attrib.c tin-2.1.1_r1/src/attrib.c
--- tin-2.1.1/src/attrib.c2012-02-21 09:42:39.000000000 +0100
+++ tin-2.1.1_r1/src/attrib.c2012-02-21 17:16:54.000000000 +0100
< at >< at > -224,9 +224,6 < at >< at > set_default_state(
 state->signature_repost = FALSE;
 state->sort_article_type = FALSE;
 state->sort_threads_type = FALSE;
-#ifdef USE_HEAPSORT
-state->so

Re: [tin 2.1.1] snaphsots

Urs Janßen — 2012-02-21T15:43:44

In <E1RzmgR-0001ad-Lr< at >akk10-int.akk.uni-karlsruhe.de>,
    Urs Janßen wrote:

settig of sort_function was not session presistent, here is the fix

=== modified file 'include/tin.h'
--- include/tin.h2012-02-21 10:13:30 +0000
+++ include/tin.h2012-02-21 15:39:26 +0000
< at >< at > -2009,7 +2009,9 < at >< at >
 /* set to (void)heapsort or qsort */
 #ifndef USE_HEAPSORT
 #define tin_sort qsort
-#endif
+#else
+#define MAX_SORT_FUNCS 1
+#endif /* !USE_HEAPSORT */
 
 /* Seperator between dir part of path & the filename */
 #define DIRSEP'/'

=== modified file 'src/config.c'
--- src/config.c2012-02-21 10:13:30 +0000
+++ src/config.c2012-02-21 15:37:13 +0000
< at >< at > -656,6 +656,11 < at >< at >
 if (match_integer(buf, "sort_threads_type=", &tinrc.sort_threads_type, SORT_THREADS_BY_LAST_POSTING_DATE_ASCEND))
 break;
 
+#ifdef USE_HEAPSORT
+if (match_integer(buf, "sort_function=", &tinrc.sort_function, MAX_SORT_FUNCS))
+break;
+#endif /* USE_HEAPSORT */
+
 if (match_integer(buf, "scroll_lines=", &tinrc.scroll_lines, 0))
 brea

Re: [tin 2.1.1] snaphsots

Urs Janßen — 2012-02-21T12:31:25

Yes on linux. The heapsort in the library uses nearly the same code as the
one provided with tin (the libs version may have an overflow problem with
large arrays as it uses int instead of size_t for several vars; I guess this
will be fixed in the next release) and I doubt that someone will optimize
that code in the (near) future. So (currently) the only benefit would be
a smaller (but less portable) binarie. If there will be some optimization
later on using the libs version may be a win...

I'm fine with the current solution - I just wanted to mention the existence
of the library. <http://libbsd.freedesktop.org/wiki/>

urs

Re: [tin 2.1.1] snaphsots

Thomas Dickey — 2012-02-21T10:56:11

agreed (cut/paste error).  It can be removed.


on Linux?  I'm using that port with Lynx, but haven't really decided
if it's a good thing to use since its headers seem to change without
much reasoning involved.  (It can be troublesome to support in different
versions).  Something to think about.

[tin 2.1.1] snaphsots

Urs Janßen — 2012-02-21T10:10:55

I've made snapshots of the upcomming tin 2.1.1 (unstable) release
including the --enbale-heapsort configure option. If specified a new
'M'enu option is available to specify the sorting function used. Tit
defaults to qsort(3) and can be changed to heapsort(3) which may
speed up sorting when the data is somewhat presorted (usualy the case
in large groups with long threads).

<ftp://ftp.tin.org/pub/news/clients/tin/unstable/snapshots/>

< at >Thomas: vsnprintf is doubled in configure.in:AC_CHECK_FUNCS() and on
debian based linux systems a useable heapsort is available in libbsd
(-lbsd /  <bsd/stdlib.h>) - I don't know if it's worth checking for
that or if "our" heapsort.c is good enough. If've changed
configure.in  and src/Makefile[.in] so heapsort.c is only compiled if
--enbale-heapsort is given (like it's done with trace.c and
--enable-trace).

urs

Re: qsort can get very slow

Thomas Dickey — 2012-02-20T11:54:52

oops - I see that with it disabled, the code still refers to heapsort.

That aspect should be recoded as a function pointer which is configurable
after building (will work on that this evening).

Re: qsort can get very slow

Thomas Dickey — 2012-02-20T10:46:46

here's the part with the option added (am starting to work on the other
part...)

Re: qsort can get very slow

Thomas Dickey — 2012-02-19T15:00:57

thanks.  I'll take a look today regarding the option and (I have as
usual some updates to configure macros which might be useful).

Re: qsort can get very slow

Urs Janßen — 2012-02-19T14:42:02

why ever the attachment was missing from the last mail, here it is...

Re: qsort can get very slow

Thomas Dickey — 2012-02-19T13:46:28

I didn't see an attachment.


sounds right...

Re: qsort can get very slow

Urs Janßen — 2012-02-19T12:17:14

With the attached patch configure looks for system heapsort() (but is
doesn't check if it is qsort(3)-api compatible) and if not found the
heapsort() from heapsort.c is used (switching back to qsort can be done
via the tin_sort degine in tin.h).

This is not well tested and should either be switched into a runtime
option or at least a configure option (--enable-heapsort or whatever)
and the configure check for a system heapsort should also check if the
one found is sort(3)-api compatible.

urs

Re: qsort can get very slow

Urs Janßen — 2012-02-18T01:45:10

fullquote as I Cc tin-dev


I don't see a big win over here (Linux on a Pentium 4) in what I
assume is normal usage (but I also see that filtering/sorting _is_
faster). quick and dirty test case:

enter a group with 88510 articles, group is read via nntp from remote
server (100mbit uplink), my normal filter file (around 50 complex rules)
is used, quit tin once all the filtering is done (press 'Q' during
filtering),

tin with qsort from libc:
= 39.065967 secs.

tin with heapsort.c from freebsd:
= 35.564494 secs.

the heapsort version is about 3.5 secs faster most of the time is
spent for fetching data via nntp.

An acceptable solution for general should IMHO look like (after someone
did some more testing with just timing the sorting with more data):

configure check for qsort compatible heapsort in libc like it
is done for qsort with CF_COMPTYPE (aclocal.m4).

if heapsort is found use that, otherwise stick to qsort.

or if we like to benefit from the speed improvement (if there is any
notable) of heapsort ov