gmane.comp.security.virus.clamav.devel

Re: Does clamav work with hex or characters?

Kaushik Vaidyanathan — 2013-03-26T01:50:15

Ah.. that help a lot Dave.

P.S. Excuse my typos. Touched and not typed.
On Mar 25, 2013 5:39 PM, "David Raynor" <draynor< at >sourcefire.com> wrote:

_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

Re: Does clamav work with hex or characters?

David Raynor — 2013-03-25T21:47:33

On Sat, Mar 23, 2013 at 3:34 PM, Kaushik Vaidyanathan <
kvaidya1< at >andrew.cmu.edu> wrote:


Read from signature, yes. Read from file, no. To quickly compare bytes it
is better to do it using the in-file binary representation. It is more
direct to say that cli_hex2str() is converting human-readable
representation of a hexadecimal number into the binary equivalent. For any
byte pattern to match, the signature-format equivalent will take twice as
many bytes as the raw binary value.

Example: "Hex" in ASCII
Actual data is 3 bytes long. 1st byte: 0x48. 2nd byte: 0x65. 3rd byte: 0x78
Signature-format equivalent is 6 bytes long, one for each hex digit.

This is where the name of the function came from. Input and output are both
char arrays (i.e. strings). The function takes in the "hex"-format version
of the content [486578], and returns the content in a usable string format
[Hex]. Hence, from "hex" to string.

Dave R.

Re: Does clamav work with hex or characters?

Steven Morgan — 2013-03-25T20:10:10

Kaushik,

Those assumptions are correct. You should also have a look at the file
libclamav/readdb.c. That is where the signatures are read and loaded into
memory for the pattern matchers.

Steve

On Sat, Mar 23, 2013 at 3:34 PM, Kaushik Vaidyanathan <
kvaidya1< at >andrew.cmu.edu> wrote:

_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

Re: clamav-devel Digest, Vol 100, Issue 6

Steven Morgan — 2013-03-25T14:59:05

Hello,

Partially, but now I realize that what you are asking is probably more
extensive than disabling llvm compilation.

Steve

On Sat, Mar 23, 2013 at 8:05 AM, G.W. Haywood <
clamav-devel< at >jubileegroup.co.uk> wrote:

_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

Re: Does clamav work with hex or characters?

Kaushik Vaidyanathan — 2013-03-23T19:34:51

Hi Matt

Thanks for your detailed explanation on how signature gets stored and
interpreted.

I was looking up the codes in libclamav to see what data formats get used
for string compare. Some backtracking from cli_bm_scanbuff took me to str.c
where I see there is a function" cli_hex2str", which if I understand
correctly maps two hexs to one character (unsigned char). Would it fair to
speculate that this function is used by the clamav engine to map two hexs
read from a signature or scanned file into one char for string matching
purposes?

Thank you..


On Sat, Mar 23, 2013 at 11:02 AM, Matt Olney <molney< at >sourcefire.com> wrote:

_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

Re: Does clamav work with hex or characters?

Matt Olney — 2013-03-23T19:23:00

It was pointed out to me that in my explanation I failed to lay out how
ClamAV avoids alerting on both executable and html files with a single
signature.  Signatures can be tagged with a target type.  A signature of
type one would only evaluate against portable executable (PE) files.  While
a signature of type three would only look at HTML files.  File types are
laid out on page 8 of the "Creating Signatures for ClamAV" document, found
here: http://www.clamav.net/doc/latest/signatures.pdf

Matt


On Sat, Mar 23, 2013 at 2:02 PM, Matt Olney <molney< at >sourcefire.com> wrote:

_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

Re: Does clamav work with hex or characters?

Matt Olney — 2013-03-23T18:02:26

Well....data is data.  There is no difference (from a storage perspective)
from an executable with an "inc ecx" instruction or a text document with an
"A".  Both are represented by the value 0x41.  So from Clam's perspective,
a signature matching a single A would be identical to a signature that
detected a single "inc ecx" instruction.  Both would look for 41.

In short your statement "some files are hex and some are character-based"
isn't really accurate.  At the risk of painting with a broad brush, I would
say that all files are stored as a series of values, a series of bytes.
 How you display them is different.  When I used 010 Editor to view a file
as hex, I get a set of ascii-hex representations.  When I look at a file
with a web-browser I get ascii text.  But underlying all of that is the
same idea, a set of bytes.  And that is how ClamAV treats all files.

A signature with a 41 in it would be converted in memory to look for 0x41,
a single byte of value 0x41.  A signature written like that would detect

Re: clamav-devel Digest, Vol 100, Issue 6

G.W. Haywood — 2013-03-23T12:05:25

Hi there,

On Sat, 23 Mar 2013, Steven Morgan wrote:

Thanks for the reply.

Does that do what I'm asking?  I've searched for any documentation and
for anything in the change logs, and I've come up with nothing to tell
me what it might do.  Can I take it that the bytecode interpreter is
this llvm thing?

--

73,
Ged.
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

Does clamav work with hex or characters?

Kaushik Vaidyanathan — 2013-03-23T00:46:58

Hi

I have a basic question. Most body-based signatures are hex based(lets
focus on fixed string signatures alone for simplicity), whereas some of the
files are hex(EXE) or character-based(HTML).

In the code I see unsigned chars used predominantly to represent patterns
and file contents. At the very core, do the string matching algorithms,
mainly extended Boyer Moore, I would like to understand how the datatypes
gets manipulated.

1) Do the character based files get translated to hex to compare with body
based signatures?

2) Does the signature get treated as a string of chars?
If yes,
Does a toy signature "fe" gets treated as two chars(8 bits each) for "f"
and "e" (or)
Does the code read the signature "fe" and maps into one character based on
the ASCII table (for example)?

Thank you..
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

Re: New Version of ClamAV

Steven Morgan — 2013-03-22T17:44:55

G.W. Haywood,

Please try running ./configure with the --disable-llvm option.

Steve

On Fri, Mar 22, 2013 at 11:47 AM, G.W. Haywood <
clamav-devel< at >jubileegroup.co.uk> wrote:

_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

Re: New Version of ClamAV

G.W. Haywood — 2013-03-22T16:47:26

Hi there,

On Thu, 21 Mar 2013, Matt Olney wrote:


I want to be able to disable compilation of the bytecode interpreter
at configure time.

--

73,
Ged.
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

Re: New Version of ClamAV

Eugene Crosser — 2013-03-20T19:07:46

[...]

That's what I have to do currently. But it's extra conversion. I have my own
MIME parser, it gives me parts of the message in the form of memory areas. It's
a shame to write the data that is already read/mmaped back to a file, even if
it's on tmpfs.

Eugene

_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

Re: New Version of ClamAV

Marc Balmer — 2013-03-20T17:02:02

Am 20.03.13 17:11, schrieb Ladar Levison:

Isn't that what most MTAs do anyway?  Receiving a message over a socket
and storing it in a file.

When I added (or rather fixed) the filedescriptor passing code in
ClamAV, I did it to optimally work with sendmail milters and it did
speed up things tremendously (before, you add to pass the whole message
content over the socket to the clamav daemon process).

If you temporarily store the message content on a ramdisk, it should get
quite fast.

(speaking about a scenario with running clamd here, btw)

_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

Re: New Version of ClamAV

Ladar Levison — 2013-03-20T16:11:29

 
I second that! Right now we have to write an email message to disk just so we can pass in a file descriptor.

I'd also advocate better support/documentation/testing of a "lite" ClamAV build configuration. Namely geared towards systems
with limited memory and/or smaller than normal thread stack sizes, or strict security environments, yet still capable of
examining large file(s). This would include being designed to run (as effectively as possible) without the builtin runtime.
Security audits have flagged ClamAV in the past because it ships with LLVM code and could (theoretically) execute arbitrary code
from a compromised definitions file.

_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

Re: New Version of ClamAV

Heinrich Schuchardt — 2013-03-20T14:57:00

Hello Matt,

with Kernel 2.6.36 the fanotify API was introduced that allows
to make ClamAV an online virus scanner. Since Kernel 3.8.0 the
major errors of the implementation have been resolved. So it
is usable now.

As a prove of concept I created a project at
https://github.com/xypron/skyldav
which I am using on my Linux computers and which already helped
me to identify viruses while opening downloaded files.

A usable application should provide the following:
- A KDE or a Gnome system tray application which notifies a user
  if a virus has been discovered and which allows to schedule
  complete system scans.
- A deamon scanning files before read and at close after write.
  This daemon should discover mount events and automatically add
  new mounts to the list of watched mounts, e.g. when a network
  drive is connected or a medium loaded.

I am aware of some coding for the fanotify API in the source
trunk but it seems to provide neither of:
- a system tray application
- the capability to watch complete mou

Re: New Version of ClamAV

Eugene Crosser — 2013-03-20T14:47:18


API to scan data that is already read (or, more practically, mmap()'ed)
into memory, a la scan_buff(void *start, size_t length, ...<options etc.>)

Thanks,

Eugene

_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

New Version of ClamAV

Matt Olney — 2013-03-20T14:35:52

Hey all,



We're currently scoping out the next version of ClamAV.  We have a number
of ideas in house, but I wanted to solicit some feedback from our users
about what you might be interested in seeing.



Before you ask, we don't have a lot of information that we're ready to
share on our end about what we're planning, so I don't want to promise
anything yet.  In general we're looking to expand the detection capability,
the engine's stability and make the system a little more usable.  As we
firm things up, we'll let you guys know more about what we're working on.



We will also be interested, as we get further down the road, in beta
testers.  I think you'll see a lot of new functionality in ClamAV and we'd
appreciate as many eyes as possible on it once we're ready to show it off.



And no, we don't have an estimated release date :)



Thanks in advance for your ideas!  Please send your ideas to this list so
we can track them.



Matt
_______________________________________________
http://lurker.clamav.net

Re: ClamAV 0.97.7 has been released!

David F. Skoll — 2013-03-19T15:45:56

From: Matt Olney <molney< at >sourcefire.com>


[...]

Thanks for those Bugzilla links.

Regards,

David.
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

Re: ClamAV 0.97.7 has been released!

Matt Olney — 2013-03-18T12:59:14

There is additional information in the bugs associated with the fixes:

https://bugzilla.clamav.net/show_bug.cgi?id=6803

https://bugzilla.clamav.net/show_bug.cgi?id=6804

https://bugzilla.clamav.net/show_bug.cgi?id=6805

https://bugzilla.clamav.net/show_bug.cgi?id=6806

https://bugzilla.clamav.net/show_bug.cgi?id=6807

https://bugzilla.clamav.net/show_bug.cgi?id=6808

https://bugzilla.clamav.net/show_bug.cgi?id=6809


Matt


On Sat, Mar 16, 2013 at 9:57 AM, David F. Skoll <dfs< at >roaringpenguin.com>wrote:

_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

Re: Contribute signatures

Joel Esler — 2013-03-17T05:42:07

I forwarded your email over to Alain.  Let me ping him.  

--
Joel Esler
Sent from my iPad 

On Mar 16, 2013, at 10:49 PM, "Andy Singer" <andy< at >orbitech.org> wrote:

_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

Contribute signatures

Andy Singer — 2013-03-17T02:49:12

Hi,
I am interested in writing signatures for ClamAV. I plan on doing this
regularly; should I email them to someone or upload them on the virus
submission page? I tried contacting Alain / Joel, but I never got a
response. I wrote a few signatures below, mostly for malware cryptors. If
you don't want to publish them, could I publish them here:
http://www.clamav.net/lang/en/download/cvd/3rdparty?

ndb
Trojan.Packed:1:EP+0:558BEC{-40}8B45??50E8????????83C4048BE55DC3
Trojan.Packed-1:1:EP+0:558BEC{-4}89????????00E8????FFFF5DC3
Trojan.Packed-2:1:*:8B2C2483C404C3
Trojan.Packed-3:1:*:54536A006A006A006A006A
Trojan.Packed-4:1:EP+0:33F6{-2}81??????400083EE??8B??FF6A
Trojan.Packed-5:1:EP+0:2BF6{-2}81??????400083EE??8B??FF6A
Trojan.Zbot:1:*:C745F801000000C745F8C10C00008B45FC05
Trojan.Zbot-1:1:EP+0:5589E583EC18C7042402000000FF1528614000E8F8FEFFFF
Rogue.Installer:1:*:8D4DE0895DE0895DE4895DE8E8160F0000568D4D8C895D8C895D90895D94E804
Trojan.Downloader.Cutwail:1:*:5B3C83C3188B531C2B55088D9B880000008B1B33C085DB74
Trojan.Banker