gmane.editors.sed.user

Re: join two files

Ruud H.G. van Tol — 2013-05-16T20:15:42

With autosplit:

     perl -awlne'
         BEGIN{%h=`cat ./file1`=~/(.*?);(.*)/g}
         print"$_ ",$h{$F[0]}||"N\/A"
     ' ./file2

The BEGIN-block parses file1 into a hash. The print-line postfixes each 
row from file2 with the extra info.

See `perldoc perlrun` about autosplit.

Re: how to duplicate "columns" in a file

Tim Chase — 2013-05-16T18:43:45

If that's all you want, it should be as simple as

  sed 's/.*/& &/' infile.txt > outfile.txt

If you have multiple columns and only want to delimit certain ones,
you'd have to provide more detail:  Are they tab-delimited vs.
space-delimited vs. comma-delimited?  Are fields quoted, and if so,
how?  Which columns need to be duplicated, and where do the
duplicated need to appear?

-tim

how to duplicate "columns" in a file

2013-05-16T18:17:15

Hi all,
 
hope you can help me with how to duplicate "columns" in a file:
 
before:
 
aaa
bbb
ccc
 
 
after:
 
aaa aaa
bbb bbb
ccc ccc
 
 
Thanks in advance, 
Magnus

Re: join two files

Jim Hill — 2013-05-16T16:52:10


I was in old-school mode, avoiding holding the entire search set in 
memory.  I can't imagine any plausible circumstance where that could be 
a problem these days.

Re: join two files

Davide Brini — 2013-05-16T16:36:52


Why not the simpler and single-pass

awk -F ' *; *' '

NR == FNR {
  a[$1] = $2
  next
}

{
  f = ( ($1 in a) ? a[$1] : "N/A" )
  print $0 " " f ";"
}

' file1 file2


I assume spacing around semicolons isn't relevant; in case it is, the
modifications are obvious (although I'd consider such an environment
slightly broken).

Re: join two files

Jim Hill — 2013-05-16T16:25:08

I brain-farted on each key lookup being against the entire selection 
set, of course it's a hash lookup. Apologies, I guess I have to correct 
"casual" to "hasty".

Re: join two files

Ruud H.G. van Tol — 2013-05-16T16:04:48


The Perl variants only reads each file's line exactly once, so is as 
optimal as it can get.

Re: join two files

Jim Hill — 2013-05-16T15:57:40

On 05/15/2013 11:51 PM, MOKRANI Rachid wrote:> Hi,
 >
 > Well this is a good solution and it work. You should send it to the
 > list so that others can benefit too.

ok, happy to.

awk -vFS=\; -vOFS=\; '{ printf("%04d;%s\n",$1,$0) }' file1>< at >file1
awk -vFS=\; -vOFS=\; '{ printf("%04d;%s\n",$1,$0) }' file2>< at >file2
join  -a1 -t\; -o 1.2,1.3,2.3 -eN/A < at >file2 < at >file1


So

- Therry Blanc's sed solution actually delivers what was requested, but 
I have to wonder if the O(NM) wouldn't start mattering even on today's 
hardware.

- Ruud H.G. van Tol's perl ... kinda shows perl's strengths, don't it? 
and it's O(lgN M),

- and this one's linear and probably easier on the uninitiated, but 
you'd have to tweak it to avoid multiple passes.

The O()'s are casual estimates, and unless this is for really 
industrial-strength data volumes or overeager optimizers like me I can't 
see them mattering much.

Re: Insert character x at...

Aurelio Jargas — 2013-05-16T14:59:39


One alternative to insert a char after a column N is to use the numeric
flag in the s command.

 $ echo 123456789 | sed 's/./&x/5'
12345x6789

Substitute any char (.) by itself (&) and an x. But only in the 5th
occurrence.

Re: Insert character x at...

Tim Chase — 2013-05-16T15:10:53

Doh, I missed the closing "\)" in there.  Should have been

  sed '/^0530/s/\(.\{31\}\)/\1x/' input.txt

which could have been reduced in this particular case to

  sed '/^0530/s/.\{31\}/&x/' input.txt


But that's pretty slick.  I don't often recall the /{number} flag, as
vi/vim doesn't offer it, so I don't have as much occasion to use it.

-tim

Re: Insert character x at...

Tim Chase — 2013-05-16T14:20:16

For the original request:

  sed '/^0530/s/\(.\{31\}/\1x/' input.txt

For the followup, you can gather the rest of the characters and juggle
them accordingly:

  sed '/^0530/s/\(.\{31\}\)\(.\{2\}\)./\1x\2/' input.txt

The 31 is your insertion-position-minus-one, and the "2" in "\{2\}" is
the number of characters you want to keep after the inserted
quantity.  If you want to insert more than one character and delete
the same number of characters, you can do it with

  sed '/^0530/s/\(.\{31\}\)\(.\{2\}\).\{3\}/\1foo\2/' input.txt

where the "3" is the length of the replacement "foo".

Adjust for any fenceposting errors you stumble across ;-)

-tim

Re: join two files

Ruud H.G. van Tol — 2013-05-16T12:26:11


Alternatives:

   perl -wpe '
       BEGIN{%h=`cat ./file1`=~/(\w+)\W+(\w+)/g}
       s/((\S+).*)/"$1 ".($h{$2}||"N\/A")/e
' ./file2

   perl -wpe '
       BEGIN{%h=`cat ./file1`=~/(.*?);(.*)/g}
       s/((\S+).*)/"$1 ".($h{$2}||"N\/A")/e
' ./file2

Re: join two files

Ruud H.G. van Tol — 2013-05-16T12:08:00

In Perl:

perl -wpe '
   BEGIN{%h=map{split/\s*;\s*/}`cat ./file1`}
   s/((\S+).*)/"$1 ".($h{$2}||"N\/A")/e
' ./file2

Num ; Comments ; Nbr_pass
1 ; AAAA ; N/A
2 ; BBBBB ; N/A
3 ; CC ; N/A
4 ; XXDF ; N/A
5 ; KIJHG ; 1
6 ; GFRD ; N/A
7 ; HGTFEDFR ; 1
8 ; DJHY ; 4
9 ; FDGRTFG ; N/A
10 ; QWXVG ; N/A
11 ; FTGJ ; N/A
12 ; DHGRF ; 1
13 ; GFDRFE ; 5
14 ; GFDRFE ; N/A
15 ; FGTF ; 7
16 ; LKJU ; N/A
17 ; DFDE ; N/A


Make it:

   /"$1 ".($h{$2}||"N\/A")." ;"/

if you need the trailing " ;".

Re: join two files

Frost — 2013-05-16T02:21:30

Hello,

Try:

catEndingFile.txt >> BeginningFile.txt

The >> will append the first file to the end of the second file. 
You can also:

cat file1.txt file2.txt file3.txt > file123.txt

Michael

Insert character x at...

blackmuerte — 2013-05-14T21:17:16

Hi all, I'm needing some help creating a command line that's hopefully simple.

I'd like to use a sed command in a batch file to do the following on a fixed format input file:

For every line in input file that begins with '0530' (in first 4 characters on line), insert 'x' at position/column 32 on same line, then output the whole file to a newfile.

Optionally, if its possible to delete one character (a blank space) on same line at position 34, that would be helpful as well, to preserve formatting.

Any/all help is greatly appreciated.

Thanks

Re: join two files

Werfgam Nadler — 2013-05-16T10:47:44

Hello,

here is a solution.

sed  '
$!{
N;
s/^/\n/;
D
}

/^\([^\n; ]*\)\( *\);.*\n\1\( *\);/!{

s/\n/ N\/A ;&/
/\n/!s/$/ N\/A;/
}

/^\([^\n; ]*\)\( *\);\([^\n][^\n]*\)\(.*\)\n\1\( *\);\([^\n][^\n]*\)/{

s//\1\2;\3 \6\5\4/;
}
P;
D;
'       file2    file1  
#  PS : it is ur file2 and file1 in that order.

this gives the solution below.




Num ; Comments ; Nbr_pass;
1 ; AAAA ; N/A ;
2 ; BBBBB ; N/A ;
3 ; CC ; N/A ;
4 ; XXDF ; N/A ;
5 ; KIJHG ; 1;
6 ; GFRD ; N/A ;
7 ; HGTFEDFR ; 1;
8 ; DJHY ; 4;
9 ; FDGRTFG ; N/A ;
10 ; QWXVG ; N/A ;
11 ; FTGJ ; N/A ;
12 ; DHGRF ; 1;
13 ; GFDRFE ; 5;
14 ; GFDRFE ; N/A ;
15 ; FGTF ; 7;
16 ; LKJU ; N/A ;
17 ; DFDE ; N/A;






________________________________
 From: MOKRANI Rachid <rachid.mokrani< at >ifpen.fr>
To: sed-users< at >yahoogroups.com 
Sent: Wednesday, May 15, 2013 5:25 PM
Subject: join two files
 

Hi,

I need some help for doing the following command with sed. Is someone
could gibe me the exemple ?

Many thanks.


=============================
cat /tmp/file_1

Num;Nbr_pass;
5;1;
7;1;
8;4;
12;1;
13;5;
15;7;
=============================

cat /tmp/file2

Num ; Comments ;
1 ; AAAA ;
2 ; BBBBB ;
3 ; CC ;
4 ; XXDF ;
5 ; KIJHG ;
6 ; GFRD ;
7 ; HGTFEDFR ;
8 ; DJHY ;
9 ; FDGRTFG ;
10 ; QWXVG ;
11 ; FTGJ ;
12 ; DHGRF ;
13 ; GFDRFE ;
14 ; GFDRFE ;
15 ; FGTF ;
16 ; LKJU ;
17 ; DFDE ;

=============================
The result, I would like to have. 


Num ; Comments ; Nbr_pass ;
1 ; AAAA ; N/A ; 
2 ; BBBBB ; N/A;
3 ; CC ; N/A;
4 ; XXDF ; N/A;
5 ; KIJHG ;1 ;
6 ; GFRD ; N/A;
7 ; HGTFEDFR ; 1 ;
8 ; DJHY ; 1 ;
9 ; FDGRTFG ; N/A;
10 ; QWXVG ; N/A;
11 ; FTGJ ; N/A;
12 ; DHGRF ; 1 ;
13 ; GFDRFE ; 5 ;
14 ; GFDRFE ; N/A;
15 ; FGTF ; 7 ;
16 ; LKJU ; N/A;
17 ; DFDE ; N/A;



Many thanks for help.
Regards.


__________________________
Avant d'imprimer, pensez à l'environnement ! Please consider the environment before printing ! 
Ce message et toutes ses pièces jointes sont confidentiels et établis à l'intention exclusive de ses destinataires. Toute utilisation non conforme à sa destination, toute diffusion ou toute publication, totale ou partielle, est interdite, sauf autorisation expresse. IFP Energies nouvelles décline toute responsabilité au titre de ce message. This message and any attachments are confidential and intended solely for the addressees. Any unauthorised use or dissemination is prohibited. IFP Energies nouvelles should not be liable for this message.
__________________________


[Non-text portions of this message have been removed]



------------------------------------

RE: join two files

MOKRANI Rachid — 2013-05-15T16:19:28

well, join don't give the good result :-( 
and same with paste, no good result.

/usr/bin/join --nocheck-order -t ';' -a 1 file1 file2
Num;Nbr_pass;;Comments;
5;1;;KIJHG;
7;1;;HGTFEDFR;
8;4;;DJHY;
12;1;
13;1;
15;7;


A sed syntax would be appreciated. (CentOS 6.3 - kernel 2.6.32 - sed release 4.2.1)

Thanks in advance. 




 

__________________________
Avant d'imprimer, pensez à l'environnement ! Please consider the environment before printing ! 
Ce message et toutes ses pièces jointes sont confidentiels et établis à l'intention exclusive de ses destinataires. Toute utilisation non conforme à sa destination, toute diffusion ou toute publication, totale ou partielle, est interdite, sauf autorisation expresse. IFP Energies nouvelles décline toute responsabilité au titre de ce message. This message and any attachments are confidential and intended solely for the addressees. Any unauthorised use or dissemination is prohibited. IFP Energies nouvelles should not be liable for this message.
__________________________

Re: join two files

Jim Hill — 2013-05-15T15:23:23

I'd use `join` to get the data itself in the right places and `sed` to do
the formatting.


[Non-text portions of this message have been removed]

Re: join two files

Thierry Blanc — 2013-05-15T14:10:50

GNU sed version 4.2.1

well, I tested it, it worked on ubuntu 12.10, kernel 3.7...


$ sed  "$(sed 's|;|s/$/|;s|;| ;/|' file1)"  file2 |sed '/;.*;.*;.*/!s|$| 
N/A;|'
1 ; AAAA ; N/A;
2 ; BBBBB ; N/A;
3 ; CC ; N/A;
4 ; XXDF ; N/A;
5 ; KIJHG ;1 ;
6 ; GFRD ; N/A;
7 ; HGTFEDFR ;1 ;
8 ; DJHY ;4 ;
9 ; FDGRTFG ; N/A;
10 ; QWXVG ; N/A;
11 ; FTGJ ; N/A;
12 ; DHGRF ;1 ;
13 ; GFDRFE ;5 ;
14 ; GFDRFE ; N/A;
15 ; FGTF ;7 ;
16 ; LKJU ; N/A;
17 ; DFDE ; N/A;



On 05/15/2013 03:47 PM, Amarendra Godbole wrote:

Re: join two files

Amarendra Godbole — 2013-05-15T13:47:59

Rachid,

Thierry may not have tested his example mostly, and hence sed version should not matter. With some tweaks, and knowledge of sed, you should be able to get it work. Thanks.

-ag

--
sent via 100% recycled electrons from my mobile command center.

On May 15, 2013, at 6:27 AM, "MOKRANI Rachid" <rachid.mokrani< at >ifpen.fr> wrote:

RE: join two files

MOKRANI Rachid — 2013-05-15T13:26:42

Thanks Thierry,

But I think we don't have the same sed version.

My release is (standard CentOS):
sed --version
GNU sed version 4.2.1


Doesn't work.


 

__________________________
Avant d'imprimer, pensez à l'environnement ! Please consider the environment before printing ! 
Ce message et toutes ses pièces jointes sont confidentiels et établis à l'intention exclusive de ses destinataires. Toute utilisation non conforme à sa destination, toute diffusion ou toute publication, totale ou partielle, est interdite, sauf autorisation expresse. IFP Energies nouvelles décline toute responsabilité au titre de ce message. This message and any attachments are confidential and intended solely for the addressees. Any unauthorised use or dissemination is prohibited. IFP Energies nouvelles should not be liable for this message.
__________________________