A page on how to use sed -- pretty rough right now. Aimed at constructing the right sed command to change all instances of <H3>Contents</H3> (ignore the <nop>s) to *Contents* on the Wikilearn web.

Aside:

• awk is the other similar utility that I was trying to remember

See AboutThesePages.

Contents

Changing the RCS "locker" on TWiki

for f in *,v; do sed 's/nobody\:/www-data\:/' $f > x; mv x $f; done

Revision to replace <H3>Contents</H3> (ignore the <nop>s) with *Contents* -- not tested -- may need to escape some characters:

for f in *,v; do sed 's/<H3>Contents</H3>/*Contents*/' $f > x; mv x $f; done

Question: Will running a "script" like the above change the owner of the file to the user running the script? (I suspect so.)

If so, I should find a way to run it as the "Apache user" -- IIRC, there is a way to do that, something like running it as a cgi script -- maybe then it would have to be in Perl.

Google Search

* sed search replace

Resources

• http://www.student.northpark.edu/pemente/sed/bookindx.txt
• http://www.student.northpark.edu/pemente/sed/
• http://www.pegasus.rutgers.edu/~elflord/unix/sed.html -- Decent tutorial.
• http://www.spring.net/yapp-bin/public/read/unix/20 -- Command line examples
• http://zez.org/article/view/65/ -- Long regular expression examples -- view the printer friendly page -- also in Perl -- Perl much faster (100:1)
• new sed "information repository"

An Example

From http://www.student.northpark.edu/pemente/sed/bookindx.txt -- some problems with this version -- see discussion there and refined version. All by Eric Pement, SED FAQ maintainer.

     # INDEXER.SED v1.0 - indexes sorted input file
     # Annotated for seders mailing list
     {               # on every line of the file...
       :loop
       $! N          # if not the last line, get the Next line
       s/^\([^;]*;\) \(.*\)\n\1 \(.*\)/\1 \2, \3/
       t loop        # if previous substitution occurred, goto :loop
       s/;/,/        # replace the semicolon with a comma
       P             # print first line of pattern buffer
       D             # delete 1st line of buffer & redo the loop
     }

Command Line Example

First

from http://www.spring.net/yapp-bin/public/read/unix/20

Topic 20 of 21 [unix]: global search and replace methods Response 5 of 6: Paul Terry Walhus (terry) * Thu, Aug 9, 2001 (08:47) * 12 lines

Search/Replace in many Files

example of how to run a search and replace through many files in UNIX. Comes in handy for situations like when Netscape Composer changes all the links to absolute rather than relative.

from the unix command prompt:

type foreach file (*.html) where *.html is the search pattern 
there will be a new prompt. Type: 
cp $file $file.orig to backup the files 
mv $file xx which moves the old files into a 'temp' file 
sed '1,$s/search/replace/g' xx > $file where search and replace are your strings. 
Note: Special characters such as / should be preceded by a \ 
end 
Once you type 'end', it will execute these commands. 

Second

The newline character in Linux is 0x0A (^J, dec 10), not 0x0D (^M, dec 13). Unfortunately, ^J seems to be one of the characters it is not possible to quote with ^V - it gets replaced by ^@, which is 0x00 (I confirmed this by doing an od of the file before and after doing the replace).

It is possible to do it using sed. Create a sed file called, for instance, t.sed, with the following contents (enter exactly as given):

s/,//g

(Note: the backslash at the end of the first line escapes the newline, so the whole file is treated as a single line)

Then run your file through sed with the following command (assuming the file is called t.t):

sed -f t.sed t.t

This will produce the output on stdout, obviously - just redirect it to the file you require (but not the original file - use a temporary file and then rename it if you want the original file changed)

(Actually, you could do it all on the command line using a command with the form:

sed -e 's/,//g' t.t

- I just prefer to do stuff in sed files so I can tweak them easier)

HTH

Spencer Collyer <spencer@lasermount.uklinux.net>

Third

From:

[Blt-newuser] the amazing sed and regular expressions (for command line afficionados)
Date: Sun, 22 Dec 2002 21:38:20 +0100
From: Kevin Pfeiffer <pfeiffer@iu-bremen.de>  (International University Bremen)

I'm familiar with regular expressions and knew that there must be an easy way but didn't want to dig out a book. A quick google search turned up an easy answer using sed and a simple loop:

for i in *.jpg; do mv -i $i `echo $i | sed 's/.*\(_.*\)/445483\1/'`; done

This takes all files in the directory ending in '.jpg' and sends them to be renamed. The regular expression matchs all text in the name up to the underscore, which, with the remainder, is captured in the memory parenthesis and then appended (using \1) to the end of the new name (the 445483). This all looks like magic but once you've spent some time with these (and maybe a little Google or O'Reilly reading over the holidays, it quickly becomes clear (though it still seems magical).

There is a lot of information online about both regexes and sed; you'll find that many clever text editors also support regexes in their search and replace functions; you can also use them in shell scripting (similar to here), with perl programming and in many other places. So if you work with text and like solving puzzles, I think it is worth spending some time to learn a little about them.

Some Perl Alternatives

from: http://archive.develooper.com/beginners@perl.org/msg10664.html

In article <20010906163445-r01010800-a3e28827-0860-0108@192.168.0.2>, dowda@coraconnectionPLEASENOSPAM.com (David Gilden) wrote:

> perl -e 'foreach $file (glob("*")) {grep '../forum_idx.pl' '../cgi-bin/forum_idx.pl');}'

you just want to go through each file and replace those instances?

perl -pi.old -e "s|../forum_idx.pl|../cgi-bin/forum_idx.pl|g" *

see the perlrun manpage for details on all of the switches.

http://www.perldoc.com/perl5.6/pod/perlrun.html

-- brian d foy <comdog@panix.com> - Perl services for hire CGI Meta FAQ - http://www.perl.org/CGI_MetaFAQ.html Troubleshooting CGI scripts - http://www.perl.org/troubleshooting_CGI.html

How About tr

tr -d '\012' < oldfile >newfile

In vi, you can use ctrl-v ctrl-m to express the newline. For other tools, such as tr, you can express the newline by using its octal value (012) and/or using '\n'.

David E. Fox
dfox@tsoftPLEASENOSPAM.com

But ^M is not the Linux (or Unix) newline character, ^J is. You got the octal value of newline correct, but not the control character it represents (and in vi using ^V^J actually produces ^@, octal 000).

Spencer Collyer <spencer@lasermount.uklinux.net>

or,

cat commaFile | tr , '\n' > newlineFile

--Jason Kary <jkary@cisco.com>

Contributors

• RandyKramer - 11 Oct 2001
• RandyKramer - 04 Apr 2002 (transferred from home TWiki)
• <If you edit this page, add your name here, move this to the next line>

Page Ratings