gmane.network.freenet.devel

Re: Statistics Project Update #4

Steve Dougherty — 2012-05-24T17:04:13

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 05/24/2012 12:48 PM, Zlatin Balevsky wrote:

Currently at each hop the HTL is capped to the maximum HTL. Is this
what you mean? I can see why you'd want to cap to something smaller then.


That's somewhat of an issue: I don't think these number-of-appearance
CDFs are a good way to show effectiveness, but I don't know how else
to visualize it.


By all means! We're planning to have another review session in
#freenet either Friday May 25th shortly after 3:15 BST or sometime
Saturday, depending on when toad_ has time. You and any others are
very welcome to join the discussion!
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQIcBAEBAgAGBQJPvmoNAAoJECLJP19KqmFu3i8QAI9zCbqpUZGZ+UzBDqYAMQya
PSdB/WOWIiuare1FfC8X/IPLC2NzPwZJNLguZJXiaRphJwDWUrKfdlcaav/F1t8U
kIQNGcdvNf1SAPpRm/dyo1U6N60bj7J3cMVN2NXWVJdi/qHZmpHj++2Nm0QrEo1g
yUmL09v+eZqCjBfDt+S/+wRuppn6zbuUgcLrTtPnlfvj3/3FKjFaJyti4f7LuN97
BzXYfydw6iQmwP6aMIPZ+f/aT7W6T7PXUclnSdQckJi5wl4eoZJ/C2g6AX4CK6Rv
R

Re: Statistics Project Update #4

Zlatin Balevsky — 2012-05-24T16:48:33

The way we dealt with this problem in Gnutella was to cap the max htl at
each hop.  Even if an attacker sent a message with very high htl each node
on the path would reduce it to a small value before forwarding.  Not sure
if this will work with Freenet.


can you tell what is the absolutely lowest htl value that will give "good
enough" performance?


We should discuss this in more detail and have more people involved before
releasing these changes.  I can see evanbd's point but the side effects of
very high htl must be taken into account as well.
_______________________________________________
Devl mailing list
Devl-RdDMkVZAZeuJnvDnx1genB2eb7JE58TQ< at >public.gmane.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: Statistics Project Update #4

Steve Dougherty — 2012-05-24T14:21:15

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 05/23/2012 10:47 PM, Zlatin Balevsky wrote:

Good point. I'm not sure what to do to improve that behavior though. I
can add some rate limiting if that looks like it'll be necessary.


Indeed it is possible to simulate, and that was the subject of my
second update on this project. [1] My main findings are here, [2]
where one can see that it's true that an HTL of 5 or 10 or so could
provide pretty good distribution already.

evanbd, my mentor for this project, suggested the maximum HTL of 50.
Here's some of his reasoning from the #freenet logs:

2012-05-09:
"So it looks to me from the graphs like HTL 20 is plenty for the new
probes. Which I take to mean we should set the default HTL as at least
30, possibly 40. Because your nice simulated graphs don't have
problematic behaviors like clustering or partitioning or whatever :)
Basically, I think we should have a fairly high *maximum* HTL (at
least 50), and have the actual HTL be a user-specified parameter."

2012-

Re: Transport Plugins Update

Arne Babenhauserheide — 2012-05-24T08:06:02

Hi Chetan,

Your changes sound good!

At Wed, 23 May 2012 22:30:40 +0530,

Thanks for the overview!


:)

No worries. I’ll be glad to see it become more active, but I’ll be
mostly disconnected for the next 10 days anyway (we moved and don’t
yet have internet in the new home).

Best wishes,
Arne

_______________________________________________
Devl mailing list
Devl< at >freenetproject.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl

Re: Statistics Project Update #4

Zlatin Balevsky — 2012-05-24T02:47:25


On a global scale, the if the rate of new probe requests is higher
than the rate at which existing ones expire the number of active
probes at any moment will not reach balance.   Higher HTL makes a ddos
against the probe mechanism easier; in this scenario the internal
limit of 5 simultaneous probes ends up assisting the attacker.

Would it be possible to simulate a single-digit HTL network?  My
intuition suggests the graph of effectiveness of probes vs. HTL has a
logarithmic shape.

Transport Plugins Update

Chetan Hosmani — 2012-05-23T17:00:40

Hello everyone,

I have been a bit busy last week, and haven't committed much code.
Here are the following refactor changes I have implemented and pushed
to "refactor-1" branch-

1. Added a separate list for transport managers. So we basically have
a list which will contain a manager for every mode. Presently it has
two - opennet and darknet.

2. NodeCrypto functions in the following way-
a. Transport managers are created in the node irrespective of whether
opennet or darknet is running.
b. NodeCrypto on creation for every mode accesses its respective
transport manager to get a list of transports.
c. It runs the method handleNewTransport to initialise a PacketMangler
and handle each transport plugin.
d. If the list of transports is empty, then it will function using the
inherent UDPSocketHandler transport and run normally.
d. When a plugin is loaded (on registering with TransportManager), the
manager checks if the corresponding mode (opennet/darknet) exists and
notifies the NodeCrypto by calling the handleNe

Re: Statistics Project Update #4

Steve Dougherty — 2012-05-22T01:41:29

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 05/21/2012 07:46 PM, Zlatin Balevsky wrote:

In what ways could I do effective rate-limiting? Would it work to keep
track of when the last request was received from each peer, and send a
rate limiting error when the last request was too recent and the last
rate error sent to that peer was long enough ago?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQIcBAEBAgAGBQJPuu7JAAoJECLJP19KqmFurfgP/1E4S/bdp9gVyUWmH02z6Ccu
BTtwoSzR9Iw2c3y1UrvLvQhinCrqmuxzG7dcidU3MmfvQvbGVOUFqZZguTGhoPcj
bjoJAZgYkwR/398c4ople2HKcyuAAbISg8HHlQ2Y+MV4XBVqaLSgncrLVbuS3Qg5
Fx+mQZ9L2OrPzTKbNvgdk+P1sc2BCo2sHp6s6gElyAuaPDsyClQKDn64qg4FfpxU
R9SweaN4H2qaGEhRjNBy4gm9tdX3Yv0Gvbkr66EkH0bbtcNHg92d6Hn3Xu/wn3Tg
gmF/p5XVLcUmgbGUkd0ziOArHGVWHBxybNB5tg3GtBC+EAA4FKgn4sP9j4rEbpKY
MQCFbbzpOQPL1uS7YGC1Ut19dfPp03GTQzAxqDrSJ0kPLjwSzHFzaPJSLEh0VJFT
ONfQxaLHgrdUyQo5RAekl6ra94TyWExTrRbMBQuvp5gTiSufvxpH3pdZRb3WJGw7
Ok+ZYlxm+K7muixOGBFCSOz1Zdx78xJ9aLqb+lZi2Bj+0WW0kRjvW9UYh4x6pg98
GMydDQosUYxHkBc

Re: Statistics Project Update #4

Steve Dougherty — 2012-05-22T00:17:25

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 05/21/2012 07:46 PM, Zlatin Balevsky wrote:

They're limited in that each node will only accept 5 simultaneous
probes. For these purposes, probes with equal randomly assigned (in
the current code; an adversary could do something different) UIDs are
considered to be the same probe.

I haven't added any rate limiting myself. If something at the
NodeDispatcher level or below has rate limiting for all messages which
pass through them then the probes would have it. In checking just now
I didn't notice any, but I don't know.


Perhaps I should add a metric like "probe requests accepted in the
past hour"? All that occurs to me is logging MHProbe at debug,
grepping the logs for "Accepting probe with uid ", counting the lines.


I don't follow. What do you mean? At each hop a probe either stays
locally (if a candidate is not accepted) or is routed to another
random node.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQIcBAEBAgAGBQJPutsVAAoJECLJP19K

Re: Statistics Project Update #4

Zlatin Balevsky — 2012-05-21T23:46:52

Are these rate-limited in any way?  Do they obey some general
rate-limiting policies together with other messages?  IMHO maximum HTL
should be single-digits or you risk flooding.  Does fred have any
metrics collection system that would allow us to detect such flooding
events?


On Sat, May 19, 2012 at 5:46 PM, Steve Dougherty <steve-kVTqj8yhOEv2eFz/2MeuCQ< at >public.gmane.org> wrote:

Re: A better FAQ

Ian Clarke — 2012-05-20T16:30:09

Well, It's a service - not software.  It does raise a good question though,
whether we can extract the questions & answers should we decide to stop
using it.  I'll ask.

Ian.

On Sun, May 20, 2012 at 9:08 AM, Luke R. <gaming4jc2-/E1597aS9LQAvxtiuMwx3w< at >public.gmane.org> wrote:

Re: A better FAQ

Luke R. — 2012-05-20T14:08:50

It does look like a reaonably nice FAQ system, but it's ashame it's not open-source/GPL.

--- On Fri, 5/18/12, Ian Clarke <ian-RdDMkVZAZeuJnvDnx1genB2eb7JE58TQ< at >public.gmane.org> wrote:

From: Ian Clarke <ian-RdDMkVZAZeuJnvDnx1genB2eb7JE58TQ< at >public.gmane.org>
Subject: [freenet-dev] A better FAQ
To: "Discussion of development of development issues" <devl< at >freenetproject.org>
Date: Friday, May 18, 2012, 12:31 PM

A friend of mine has created this service: http://helpjuice.com/
It's basically a very smart FAQ that is capable of learning.  Our current FAQ, I think, is somewhat lackluster and intimidating.  My friend has offered to let us use HelpJuice for free - thoughts?  We should be able to integrate it into our site reasonably seamlessly.

Ian.

Statistics Project Update #4

Steve Dougherty — 2012-05-19T21:46:12

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I've finished a proof of concept for Metropolis-Hastings corrected
probes in Fred. If you are so inclined, please offer feedback on the
patches. [1] Metropolis-Hastings correction was described previously,
[2] and the probes are exposed over FCP. [3] An originating node starts
a probe request by specifying the type of result they're looking for,
and optionally a value for hopsToLive, which currently defaults to its
maximum of 50 if not specified. We will in practice initially use
something closer to 30. One can disable responding to any number of
result types on the Core settings page.

Possible types are:

BANDWIDTH - outgoing bandwidth limit.
BUILD - Freenet build number. (For instance, currently 1407.) Useful
        for measuring update propagation.
IDENTIFIER - endpoint identifier. Useful for improving estimates of
             network size and churn.
LINK_LENGTHS - differences in location between endpoint and each of
               endpoint's peers.
STORE_

A better FAQ

Ian Clarke — 2012-05-18T16:31:28

A friend of mine has created this service: http://helpjuice.com/

It's basically a very smart FAQ that is capable of learning.  Our current
FAQ, I think, is somewhat lackluster and intimidating.  My friend has
offered to let us use HelpJuice for free - thoughts?  We should be able to
integrate it into our site reasonably seamlessly.

Ian.

Re: Transport Plugins (Update)

Marco Schulze — 2012-05-16T08:44:47

Yes. Thank you.

On 15-05-2012 12:42, Chetan Hosmani wrote:

Re: Transport Plugins (Update)

Chetan Hosmani — 2012-05-15T15:42:12

On Tue, May 15, 2012 at 8:39 PM, Marco Schulze
<marco.c.schulze-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org> wrote:

Well I am not the best person to answer all of those questions.
If you check my previous two posts on devl, it explains how streams
will be supported, and what power the transport will have.

But to summarize-
1. Simply put the plugin will not deal with any of this, or at least
the plugin developer will not have to worry about messages.
2. The plugin will deal mainly with setting up the low level
sockets/streams, and returning them to fred
3. Plugin will deal with parsing the data to be sent, adding headers,
etc. This is to allow stenography.
This the plugin will implement by copying pipestreams from fred,
encoding appropriately, and sending it on its own streams.

Plugin will not perform any cryptography or tracking
messages-in-flight as such. toad's idea is to make it as simple as
possible. I will implement a TCP plugin only to demonstrate it.

I have two parts here - stream support and extendin

Re: Transport Plugins (Update)

Marco Schulze — 2012-05-15T15:09:47

What about changes to the FNP when stream transports are used? How much 
power will transport plugins have on the actual bytes being sent? For 
example, it might be advantageous for the TCP plugin to use some kind of 
PAUSE-MESSAGE marker to allow messages with higher priority to be sent 
as soon as they are queued, instead of buffering/fragmenting/packing 
messages for pack transports.

On 15-05-2012 06:20, Chetan Hosmani wrote:

Re: Transport Plugins (Update)

Chetan Hosmani — 2012-05-15T09:20:28

On Tue, May 15, 2012 at 1:44 PM, Marco Schulze
<marco.c.schulze-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org> wrote:

Thank you!
Actually as proof of concept I ll implement a simple TCP plugin. Also
as a long term goal I'll make existing UDP as a plugin.

Re: Transport Plugins (Update)

Marco Schulze — 2012-05-15T08:14:45

Nice! Which transport plugins you are planning to implement?

On 15-05-2012 04:13, Chetan Hosmani wrote:

Transport Plugins (Update)

Chetan Hosmani — 2012-05-15T07:13:33

Hello,

Sorry for the previous very lengthy update. I'll keep this one short.

After several discussions the basic idea of how I'll implement
transports is now clear.
I have begun writing some basic plugin architecture. The same has been
committed to the branch "refactor-1". I intend to finish, by the end
of this week, refactoring portions of code from NodeCrypto. The main
objective is to remove the "dependence on a single socket object"

Currently I have submitted code on the following new classes-

https://github.com/chetanhosmani/fred-staging/tree/refactor-1

1. TransportManager - This will now contain a list of available
transports (packet transports and stream transports). Transports
register here to let the node know of its availability.
2. TransportPlugin - A base class for the transports
3. StreamTransportPlugin - Base class for stream based transports. It
mainly supports a "connect" method and a "listen" method.
4. PacketTransportPlugin - This is based on PacketSocketHandler itself.

One issue I had

Re: Lantern

Arne Babenhauserheide — 2012-05-14T00:38:58

Am Sonntag, 13. Mai 2012, 13:22:57 schrieb Steve Dougherty:

I first wondered about your comment, but now I tried the program myself.

Who in his right mind writes an anonymizing program which tells Google that it 
wants to connect to your friends? Google of all the services… The only way to 
make that worse would be to use Facebook instead… 

Best wishes,
Arne

Re: Lantern

Arne Babenhauserheide — 2012-05-14T00:09:05

Am Sonntag, 13. Mai 2012, 17:38:45 schrieb Matthew Toseland:

Almost everybody I know who is serious about email uses a desktop app for 
that… 

Also I use identi.ca over choqok or emacs (identica-mode). 
And IRC over Quassel. Or emacs :) (erc)

When I’m at my own computer, no web-based program beats a desktop program.

My Parents use desktop-based programs, too.

Still I like the webinterface of freenet, because I can just send people links 
to http://127.… to show them how to configure something.

But a desktop program which uses freenet as backend could be quite a bit 
nicer. Creating a good desktop GUI is a hell of a lot of work, though.

Best wishes,
Arne
--
singing a part of the history of free software: 

- http://infinite-hands.draketo.de

_______________________________________________
Devl mailing list
Devl-RdDMkVZAZeuJnvDnx1genB2eb7JE58TQ< at >public.gmane.org
https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl