gmane.org.user-groups.linux.cplug.general

Re: Slightly OT: which software to use to enable text searching of existing pdf files?

Chris Moates — 2012-05-24T00:04:05

When you say "Supported by the document" what you mean is that the PDF 
is a text file with layout information, and maybe some illustrations or 
what have you. What you have is a bunch of JPG scans of pages of a 
manual, glued together into a multi-page JPG, essentially, by the PDF 
standard. I have this same issue in my hobbyist field, pinball machines, 
where old manuals are scanned in and bound as PDF.

So yes, you're going to require OCR software. I did a quick Google 
search and came up with this:

http://blog.konradvoelkel.de/2010/01/linux-ocr-and-pdf-problem-solved/

It sounds like what you're looking for.

Cheers,
Chris

On 5/23/2012 7:22 PM, keitho-aiW8oOy0A7DIrURfT66hzQ< at >public.gmane.org wrote:

Re: Slightly OT: which software to use to enable text searching of existing pdf files?

2012-05-23T23:22:43

Thanks for all your replies guys.

I typically use XPDf. Like pdftotext, pdfgrep, and Evince, AFAIK they all
depend on text-searching being already "supported by the document". Also,
they are useless if you don't have the owner password for those pdf files
which have been protected in some way.

I was hoping there would be a way to take an existing pdf file from an
unknown origin and with various unknown protections and somehow re-scan it
with OCR type software to create a text-searchable pdf file (not a text
file).

I guess that just isn't possible at this time with the way the pdf format
has evolved. Like I said, I don't know the technology.

Keith

Help building Linux WorkStation

Steve Fosburg — 2012-05-23T23:15:08

Is there a 'cplug' member who would advise and build a linux workstation 
for me?

My interest are; 3D CAD, 3D animation and rendering, Geographic 
Information Systems and audio/video production.

I'm retired and graphics are a strong interest/hobby.  I live in the 
Carlisle area and would like to find someone reasonably close to work on 
this project.  I would buy the computer components and pay for the 
computer assembly.

Thank you,
Steve Fosburg
sbfosburg-PkbjNfxxIARBDgjK7y7TUQ< at >public.gmane.org

Re: Slightly OT: which software to use to enable text searching of existing pdf files?

Matthew Gillespie — 2012-05-23T13:41:51

I actually just switched from XPDF to Evince (which I like much better). My understanding on XPDF if that it's become an "unmaintainable mess" [0], and it's been pulled from the Gentoo Portage Tree for that reason.

Matthew Gillespie

[0]http://dilfridge.blogspot.com/2012/02/what-about-my-precious-xpdf.html

On 05/22/12 20:44, Bret Fledderjohn wrote:

Re: Slightly OT: which software to use to enable text searching of existing pdf files?

Bret Fledderjohn — 2012-05-23T00:44:41

http://www.foolabs.com/xpdf/about.html. Xpdf seems to work well too.

On Tuesday, May 22, 2012, Logan Kennedy wrote:

Re: Slightly OT: which software to use to enable text searching of existing pdf files?

Logan Kennedy — 2012-05-23T00:22:59

Pdf2text might be useful
On May 22, 2012 7:56 PM, "Bob Igo" <bob-VHaslujj8MM< at >public.gmane.org> wrote:

Re: Slightly OT: which software to use to enable text searching of existing pdf files?

Bob Igo — 2012-05-22T23:55:41

Christian already replied with some useful info, but I had a small bit
to add.

On 05/22/2012 07:20 PM, keitho-aiW8oOy0A7DIrURfT66hzQ< at >public.gmane.org wrote:
As I vaguely recall, PDF text is basically a vector drawing of
characters, which is why/how you can zoom in to an arbitrary level and
still see no pixellation.  Searchable PDF text requires a text index, as
Christian alluded.  You won't get that from a PDF that is basically a
JPG with a PDF wrapper (from a scanner) unless the process did some OCR,
also as Christian alluded.

Gtg :)

Re: Slightly OT: which software to use to enable text searching of existing pdf files?

Christian Pearce — 2012-05-22T23:37:55

* Alfresco -- heavy weight

Are the PDFs scans orbuilt from text.  That makes a big difference in how
you can do the searching.  If they are not built from text, you need to
pass it through some sort of OCR scanner to get the words.  That just made
it a whole lot more complicated. At that point I would consider trying to
publish it and have google index it.  Then search your site with google.
 If the material isn't copyrighted.

On Tue, May 22, 2012 at 7:20 PM, <keitho-aiW8oOy0A7DIrURfT66hzQ< at >public.gmane.org> wrote:

Slightly OT: which software to use to enable text searching of existing pdf files?

2012-05-22T23:20:19

As a hobby, I have been working on old analog Tektronix scopes. It's
generally easy to download (free) the service manuals in pdf format. They
are often over 400 pages.

Unfortunately, the good people who have made these free pdf files
available for whatever reason have almost never made them text searchable.
Actually, often even the manuals you buy aren't text searchable.

Is there a Linux-based utility or software which can take an existing pdf
file and generate a new version which is text searchable? Regardless of
whatever security or restrictions the original producer built into the pdf
file?

I don't know anything about pdf file systems or structure, so maybe this
is a dumb question. But if it were possible using Linux I'd sure like to
know how. Also, I really do not want to have to use Adobe software for
anything, if at all possible...

Thanks,
Keith Ostertag

Re: minicom, usb

Roy J. Tellason, Sr. — 2012-05-22T13:29:44

The other devices are a keyboard,  mouse,  occasionally an external drive,  or a camera.  It *is* showing up there now,  so if I can count on it continuing to do so then cool,  I'm all set with what I need to do here...

Re: minicom, usb

Roy J. Tellason, Sr. — 2012-05-22T13:27:37

Okay,  I plugged it in just now and got the following showing up in /var/log/messages:

May 22 09:17:34 kernel: usb 2-1.4.2: new full speed USB device using ehci_hcd and address 9
May 22 09:17:34 kernel: usb 2-1.4.2: configuration #1 chosen from 1 choice
May 22 09:17:34 kernel: ftdi_sio 2-1.4.2:1.0: FTDI USB Serial Device converter detected
May 22 09:17:34 kernel: drivers/usb/serial/ftdi_sio.c: Detected FT232RL
May 22 09:17:34 kernel: usb 2-1.4.2: FTDI USB Serial Device converter now attached to ttyUSB0

I tried putting /dev/ttyUSB0 in minicom just now,  and firing it up without the -s it seems to come up okay.

This should get interesting,  when I get that console cable plugged in to the serial adapter...    :-)

Re: minicom, usb

Chris Moates — 2012-05-21T23:46:14

Which USB device it technically is, and which device it presents as 
(which will be /dev/ttyUSB*) are not related. You're only interested in 
the latter, and as long as you don't have other serial devices, this 
really won't change. It'll always be /dev/ttyUSB0.

I'd poke around in the device mapping stuff like udev or the like. 
Perhaps it's blacklisted or not enabled or similar.

Cheers,
Chris

On 5/21/2012 12:59 PM, Roy J. Tellason, Sr. wrote:

Re: network video recorder (NVR) recommendation

Doug Warner — 2012-05-21T19:13:23

I've recently been setting this up at home with the cheap Chinese-made Foscam
cameras.  There's a bunch of manufacturers that use the same reference design
(Foscam, Apexis, Wansview, Loftek).

I'm currently using Zoneminder to do the digital archiving part.  It's an okay
solution, but kinda clunky.  It requires a lot of fiddling.

So far my biggest problem with these cheap cameras is the outdoor one that
faces my driveway (Foscam FI8905w) has issues with it's auto-gain control
where it will flicker/strobe when the driveway gets too bright.  This blows up
any automatic motion detection you have since essentially the entire screen is
changing.

-Doug

Re: minicom, usb

Matthew Gillespie — 2012-05-21T17:09:11

Hey Roy,

    You might try pulling out the serial adapter out and putting it back in... run dmesg and see what's in the kernel ring buffer... typically you'll see something like this:

usb 6-1: new full speed USB device number 2 using uhci_hcd
usbcore: registered new interface driver usbserial
USB Serial support registered for generic
usbcore: registered new interface driver usbserial_generic
usbserial: USB Serial Driver core
USB Serial support registered for ch341-uart
ch341 6-1:1.0: ch341-uart converter detected
usb 6-1: ch341-uart converter now attached to ttyUSB0
usbcore: registered new interface driver ch341

    Earlier you posted only 2 lines generated by the kernel, methinks there's likely more which may be revealing.

Matthew Gillespie

On 05/21/12 12:59, Roy J. Tellason, Sr. wrote:

Re: minicom, usb

Roy J. Tellason, Sr. — 2012-05-21T16:59:21

Yes,  it seems to be.  Which is the proper device is the issue here,  particularly the way USB devices are dynamically assigned -- if I find out which one it is now that's one thing,  but I have other stuff plugged in to my laptop here and when I take it out with me that stuff stays behind...

Re: minicom, usb

Matthew Daubenspeck — 2012-05-21T16:46:03

Also, once you find the proper device, make sure you let minicom know
where it is. minicom -s will run the setup, you can choose the proper
/dev/ device and save it. Sounds like it is currently configured to
/dev/modem?

Re: minicom, usb

Roy J. Tellason, Sr. — 2012-05-21T16:39:14

There's a bunch of /dev/usbdev*,  /dev/usbdev1.1_ep00,  and /dev/usbdev1.1_ep81 entries in there,  no /dev/ttyUSB* at all.

How do I know which one corresponds to the adapter?


Yup!

 
I've seen lots of earlier references to building in kernel support for different things,  and then finding out that modules worked well.  Right now lsmod is showing me the modules ftdi_sio and usb_serial used by ftdi_sio so that part seems to be working okay.  It's just the parts I'm missing here...

Re: minicom, usb

Chris Moates — 2012-05-21T16:06:36

The FTDI stuff is well supported under Linux. Look for a /dev/ttyUSB* to 
exist corresponding the adapter. Typically /dev/ttyUSB0. If you're still 
using Slackware, you might have to build in kernel support for USB 
Serial devices, I have no idea.

Cheers,
Chris

On 5/21/2012 9:20 AM, Roy J. Tellason, Sr. wrote:

Re: network video recorder (NVR) recommendation

Bob Igo — 2012-05-21T13:57:55

The devil is very much in the details, but you could roll your own with
VLC.  I've used VLC to make a commercial product that did this as one of
its components.  Take care to choose the right parameters that will let
your recorded video files 1) survive network outages (not all codecs are
truncation-resistant), 2) be of the desired quality, and 3) take up an
appropriate amount of bandwidth and disk space.

I've used Axis cameras for this.  They are pricey but robust.  Most of
them have a powerful HTTP-based API (so, cross-platform).

Re: network video recorder (NVR) recommendation

Eric C. Ross — 2012-05-21T13:50:46

Axis Communications (www.axis.com) has been making network video cameras 
and related equipment for years.  They used to make print servers many 
years ago.  I haven't used them for cameras but their print servers 
worked well and, I believe, were based on Linux or BSD.  Their print 
servers were Linux/Unix/BSD-friendly and they had pretty good support 
for it.


Eric Ross


On 5/21/2012 9:27 AM, Lee Zimmerman wrote:

network video recorder (NVR) recommendation

Lee Zimmerman — 2012-05-21T13:27:46

Hello CPLUG,

Has anyone used a Linux base network video recorder (NVR) to
record surveillance video that they would recommend?
I would be looking for something that could scale to 100 cameras.
Also any recommendation's IP camera brands.

Thanks,