gmane.network.flow-tools.user

Re: http flow request not displaying flows

Joe Loiacono — 2013-05-07T15:09:34

flow-tools-bounces-ojNDMRNHqGVygxfI3sfyqtHuzzzSOjJt< at >public.gmane.org wrote on 05/06/2013 04:49:07 PM:


Which web app software are you running? Is it possible the reboot has now
put your server in a different time zone than your devices exporting
netflow? That can cause what you're seeing.

Do a:

flow-print -f5 < ft-v05.2013-05-07.084501+0000 > ~/temp.out

... and compare time-stamps of flows to file creation time.

Joe

Re: http flow request not displaying flows

Craig Weinhold — 2013-05-07T12:29:45

Flow-tools contains a flow collection server and set of command-line utilities, but no web interface. Thus, you must have some other software package providing the web interface (flowscan, flowviewer, webview, or a custom one).

Begin by watching the apache error log (/var/log/apache2/error_log) while you make an http request. The log may provide enough information to fix the problem (e.g., "permission denied").  If it doesn't, it'll at least tell you which script is backfiring and you can figure out where the script came from and how to get support on it. Perhaps it depends on a backend database that failed to start with the reboot.

-Craig


On Mon, 6 May 2013, Kirk Olson wrote:

http flow request not displaying flows

Kirk Olson — 2013-05-06T20:49:07

In March I rebooted the Ubuntu system running the flow-tools installation.
After the reboot an http request for flow information returns no data
although the apache service seems to be running fine. I can also see flows
being captured in the /var/netflow directory.

I am sure this is more of an Ubuntu admin question so I apologize for my
ignorance. Does anyone have any ideas?

Kirk Olson
_______________________________________________
Flow-tools mailing list
flow-tools-PZzQvgnt7zHEueBKFXcDjA< at >public.gmane.org
http://mailman.splintered.net/mailman/listinfo/flow-tools

Strange ip addresses in flow_capture log files

Ivan Korjavin — 2013-03-28T06:51:28

On my freebsd box:

#uname -rimp
9.1-STABLE amd64 amd64 GENERIC
flow_tools:

Information for flow-tools-0.68_7:


Collector is ng_flow, started with

    /usr/sbin/ngctl mkpeer ipfw: netflow 30 iface0
    /usr/sbin/ngctl name ipfw:30 netflow

    /usr/sbin/ngctl msg netflow: setdlt {iface=0 dlt=12}
    /usr/sbin/ngctl msg netflow: setifindex {iface=0 index=5}
    /usr/sbin/ngctl msg netflow: settimeouts {inactive=15 active=150}
    /usr/sbin/ngctl mkpeer netflow: ksocket export inet/dgram/udp
    /usr/sbin/ngctl msg netflow:export connect inet/127.0.0.1:9995
And ipfw rule:

02750  59239017674  33111253913522 ngtee 30 ip from any to any via em0
Exported with flow_fanout for flow_capture.

# ps axww | grep flow
15106 ??  Ss        2:50,08 /usr/local/bin/flow-fanout -p
/var/run/flow-capture/flow-fanout.pid 127.0.0.1/0.0.0.0/9995
127.0.0.1/127.0.0.1/9556
16367 ??  Ss       11:28,63 /usr/local/bin/flow-capture -n 95 -N 3 -z
5 -S 5 -E270G -w /var/netflow -p
/var/run/flow-capture/flow-capture.pid 127.0.0.1/0.0.0.0/

Flowdumper not working after installing Cflow

Carlos Contreras — 2012-12-12T01:20:58

Hi,

I have installed cflow according to the instructions on the flow-tools 
README file however despite I have several flow files, I am not able to 
get any output at all using the flowdumper program, I have included the full compile process and the actual test so if you can advice me about the possible problem

I have captured the install of Cflow in case you see any errors on the make process

root< at >carlos-VirtualBox:/home/carlos/Downloads/Cflow-1.053# perl Makefile.PL CCFLAGS='-DOSU' LIBS='-lft'
Found flow-tools... using "-DOSU -I../../lib -I../../lib/.. -L../../lib -lft -lz".
Note (probably harmless): No library found for -lft
Writing Makefile for Cflow
Writing MYMETA.yml
root< at >carlos-VirtualBox:/home/carlos/Downloads/Cflow-1.053# locate lft
/etc/bash_completion.d/lftp
/usr/lib/udisks/udisks-helper-ata-smart-selftest
/usr/share/locale-langpack/en_AU/LC_MESSAGES/lftp.mo
/usr/share/locale-langpack/en_GB/LC_MESSAGES/lftp.mo
/usr/share/locale-langpack/zh_CN/LC_MESSAGES/lftp.mo
root< at >carlos-VirtualBox:/home/car

(no subject)

Amine Mouadden — 2012-12-04T14:52:09

http://beautybodystyle.com/blog/wp-content/yepyepop.php       _______________________________________________
Flow-tools mailing list
flow-tools-PZzQvgnt7zHEueBKFXcDjA< at >public.gmane.org
http://mailman.splintered.net/mailman/listinfo/flow-tools

Good bye, traditional netflow

Craig Weinhold — 2012-10-26T03:34:15

If you use Cisco products, you may be interested that they've announced the impending end for "traditional netflow" on the ASR1000 line of routers.  Here is the announcement, which includes a link to a whitepaper for migrating traditional to flexible netflow:

  http://www.cisco.com/en/US/prod/collateral/routers/ps9343/eol_C51-718332.html

Traditional netflow is what you have if your interfaces have any of these commands:

 ip route-cache flow
 ip flow ingress
 ip flow egress

The announcement only affects the ASR 1000 series of routers right now, but it points out that newer platforms like the Catalyst 6500/Sup2T, the Catalyst 4500/Sup7E, and the Catalyst 3850 are exclusively supporting flexible netflow. I suspect that the next generation of ISR routers follows suit.

-Craig

Re: Netflow v5 pretty much dead what is everyonemigrating to?

Jeroen Massar — 2012-10-08T19:24:48

[..]

effectively:

cflow == NetFlow v5
jflow == NetFlow v9

But due to licensing/naming/tradermarks/patents they are not called that
way.

But we got IPFIX now to solve that part of the hassle.

Greets,
 Jeroen

Re: Netflow v5 pretty much dead what is everyonemigratingto?

Joe Loiacono — 2012-10-08T18:55:34

FlowViewer version 4.0 will work with IPFIX/v9 provided you can get a copy 
of SiLK v3.0.

SiLK has been developed by the NetSA group at Carnegie Mellon and it's 
previous versions (prior v3.0) are freely available.

The manager of the SiLK software, Chris Inacio, has told me in emails he 
is getting all of the v3.0 approvals together, but hasn't finalized it.

I believe if you work for a federal agency or are aligned with a US 
research organization (University?) you can get a copy of v3.0 now.

The SiLK software is excellent and worth the wait, but I understand 
network management pressures to move on this.

We've used a free-ware Inmon software tool to convert sflow to netflow v5 
with success.

I've received netflow from Juniper routers and store it with flow-tools 
for years. Is that the same as "jflow"?

Best,

Joe



From:   Drew Weaver <drew.weaver< at >thenap.com>
To:     "flow-tools< at >list.splintered.net" <flow-tools< at >list.splintered.net>
Date:   10/08/2012 12:14 PM
Subject:

RE: Netflow v5 pretty much dead what is everyonemigratingto?

Volk,Gregory B — 2012-10-08T16:18:49

NFDUMP seems to be alive and up to date but I have not had much time to sit down and actually use it in a significant manner.

http://nfdump.sourceforge.net/

From: flow-tools-bounces-ojNDMRNHqGVygxfI3sfyqtHuzzzSOjJt< at >public.gmane.org [mailto:flow-tools-bounces-ojNDMRNHqGVygxfI3sfyqtHuzzzSOjJt< at >public.gmane.org] On Behalf Of Drew Weaver
Sent: Monday, October 08, 2012 11:14 AM
To: flow-tools-ojNDMRNHqGVygxfI3sfyqtHuzzzSOjJt< at >public.gmane.org
Subject: [Flow-tools] Netflow v5 pretty much dead what is everyone migrating to?

With Netflow v5's life shortening every day what software package are people who are using flow-tools switching to?

It seems like Netflow v9, ipfix, jflow and sflow are all viable technologies; is there a package like flow-tools that has collector and tools that works with all of those?

Thanks,
-Drew

If you are not the intended recipient of this message (including attachments) or if you have received this message in error, immediately notify us and delete it as well as any attachments.

Netflow v5 pretty much dead what is everyone migratingto?

Drew Weaver — 2012-10-08T16:13:35

With Netflow v5's life shortening every day what software package are people who are using flow-tools switching to?

It seems like Netflow v9, ipfix, jflow and sflow are all viable technologies; is there a package like flow-tools that has collector and tools that works with all of those?

Thanks,
-Drew

_______________________________________________
Flow-tools mailing list
flow-tools-PZzQvgnt7zHEueBKFXcDjA< at >public.gmane.org
http://mailman.splintered.net/mailman/listinfo/flow-tools

Re: how get flow fith flow-nfilter for incoming andoutgoing traffic per some net?

Konstantin V. Krotov — 2012-07-23T11:02:00

20.07.2012 16:43, Michael W. Lucas пишет:
Thanks, I found the decision to use "or" design in filter-defination

filter-definition ip-addr
     match ip-destination-address filter-primitive1
     or
     match ip-source-address filter-primitive1

filter-primitive filter-primitive1
     type ip-address-prefix
     permit xx.xx.xx.xx
     default deny

Re: how get flow fith flow-nfilter for incoming andoutgoing traffic per some net?

Joe Loiacono — 2012-07-20T13:30:30

Konstantin,

You could also let FlowViewer handle this for you in the user interface. 
You can graph it and track it (via rrdtool) as well.

http://ensight.eos.nasa.gov/FlowViewer/

Joe Loiacono




From:   "Konstantin V. Krotov" <kkv-SNifSdt4gYYox3rIn2DAYQ< at >public.gmane.org>
To:     flow-tools-ojNDMRNHqGVygxfI3sfyqtHuzzzSOjJt< at >public.gmane.org
Date:   07/20/2012 07:09 AM
Subject:        [Flow-tools] how get flow fith flow-nfilter for incoming 
and outgoing traffic per some net?
Sent by:        flow-tools-bounces-ojNDMRNHqGVygxfI3sfyqtHuzzzSOjJt< at >public.gmane.org



hello, list!
Plese, give:
how get flow fith flow-nfilter for incoming and outgoing traffic per 
some net for one request?
like nfdump tool:
nfdump 'ip xx.xx.xx.xx'?

Re: how get flow fith flow-nfilter for incoming andoutgoing traffic per some net?

Michael W. Lucas — 2012-07-20T12:43:08

Write a custom report. I put an example in:

http://blather.michaelwlucas.com/archives/494

You can also do a custom report that takes an IP address variable on
the command line.

==ml

On Fri, Jul 20, 2012 at 03:08:46PM +0400, Konstantin V. Krotov wrote:

how get flow fith flow-nfilter for incoming and outgoing traffic per some net?

Konstantin V. Krotov — 2012-07-20T11:08:46

hello, list!
Plese, give:
how get flow fith flow-nfilter for incoming and outgoing traffic per 
some net for one request?
like nfdump tool:
nfdump 'ip xx.xx.xx.xx'?

Re: Where did the community go?

Drew Weaver — 2012-04-12T18:28:16

Would be great if there was an intermediate daemon that could grab v9/ipfix and convert it to v5 at least for now :)

Craig Weinhold <craig.weinhold-22trykYRFPg< at >public.gmane.org> wrote:


Netflow v9 / IPFIX is not going to be added to flow-tools without a near-complete rewrite.

The problem with the v5 paradigm is that everything is built around fixed length records with 7-tuple keys (src/dst ip, src/dst port, protocol, tos, and input interface) and simple data fields (bytes, packets, start time, end time, etc).  Unfortunately, silk, flowd, and (I think) nfdump all continue with this paradigm. They may have added IPv6, but they basically toss out other v9/IPFIX fields. You see, V9 is open-ended with both its keys and its data fields. Think RADIUS vendor-supported attributes. The record length of each flow is not fixed, and the interpretation of flow data changes dramatically from device-to-device and config-to-config.

For example, Cisco ASA firewalls use Netflow v9 for logging both blocked and allowed traff

Re: Where did the community go?

Jaime Nebrera — 2012-04-12T17:56:42

  Hi Craig,

  I believe nprobe running as collector should do all this through
Netflow v9 templates

  Is the one we will use for our project

Jaime Nebrera
Enviado / Sent iPhone

El 12/04/2012, a las 18:58, Craig Weinhold <craig.weinhold-22trykYRFPg< at >public.gmane.org> escribió:

Re: Where did the community go?

Jaime Nebrera — 2012-04-12T17:54:32

  Hi Michael,

Jaime Nebrera
Enviado / Sent iPhone


  We can't make any code available just yet as we dint start the development :)

  Let me explain a bit:

  By end of month we will release to public the open source framework
we have done to manage snort deployments (RoR based)

  After that we will start a new project, a replacement of our current
netflow collector (java + flowtools based)

  What we would like to achieve is to get a group of funders for such
development as well as developers, designers, community interested in
participating. The project will be done even without funders but of
course would be easier if we have them :D

  The backend would be nprobe (www.ntop.org) based. Sadly this great
software has two weak points: lack of a proper web interface and
support fir the wrong nosql technology (fastbit)

  Thus the project would be about adding support to some other noSQL
engine on nprobe (hbase, Cassandra, hypertable we don't know yet) and

  To develop a new modern web based interface for

Re: Where did the community go?

Craig Weinhold — 2012-04-12T16:58:44

Netflow v9 / IPFIX is not going to be added to flow-tools without a near-complete rewrite.

The problem with the v5 paradigm is that everything is built around fixed length records with 7-tuple keys (src/dst ip, src/dst port, protocol, tos, and input interface) and simple data fields (bytes, packets, start time, end time, etc).  Unfortunately, silk, flowd, and (I think) nfdump all continue with this paradigm. They may have added IPv6, but they basically toss out other v9/IPFIX fields. You see, V9 is open-ended with both its keys and its data fields. Think RADIUS vendor-supported attributes. The record length of each flow is not fixed, and the interpretation of flow data changes dramatically from device-to-device and config-to-config.

For example, Cisco ASA firewalls use Netflow v9 for logging both blocked and allowed traffic, but you need to be able to see the extra fields to determine which. I think nfdump has some hacks to handle this.

Or, consider a router sending duplicate flows (e.g., "ip flow ingress

Re: Where did the community go?

Michael W. Lucas — 2012-04-12T16:15:55

Hi,

Yes, I saw it, but it's hard to comment without the actual code. :-)

You should send an announcement when you have something we can
install.

Thanks,
==ml

Re: Where did the community go?

Jaime Nebrera — 2012-04-12T15:42:49

   Hi Michael,


   We are about to start such project very soon.

   I dont know if my first email went through to the list but the idea 
would be to:

   * Netflow v5, v9, sFlow and IPFIX compatibility
   * noSQL backend (provably Hypertable based)
   * Web front end (RoR)
   * For sure, all open source

   For sure, all open sourced and even supported commercially. This 
project will go along the first one done for Snort that will be released 
to public April 24th (more or less)