Replace the characters in every line of a file, if it match a condition - Unix

Tag: unix,replace,sed,match,condition

I have a file "dummy" in which: If 15 character of the file matches with "R" and 28 character of the file matches with "D" then 53-56 characters should be replaced with 0.

I have tried using the below script, but it's not working.

for i in `cat dummy`
do
    if [[ `echo $i | cut -c15` = "R" ]] && [[ `echo $i | cut -c28` = "D" ]]
then
   sed -e 's/./0/53' -e 's/./0/54' -e 's/./0/55' -e 's/./0/56'
fi
done

Input File: dummy

05196220141228R201412241308D201412200055SA1155WE130800031300SL07Y051
05196220141228R201412241308A201412220350SA0731SU1950LAX C00020202020
05196220141228R201412241308D201412200055SA1155WE130823455300SL07Y051
05196220141228N201412241308A201412240007TU0548WE1107MEL C00000000015
07054820141228N201412220850D201412180300TH1400MO085000040300UL180001

Output should be:

05196220141228R201412241308D201412200055SA1155WE130800001300SL07Y051
05196220141228R201412241308A201412220350SA0731SU1950LAX C00020202020
05196220141228R201412241308D201412200055SA1155WE130800005300SL07Y051
05196220141228N201412241308A201412240007TU0548WE1107MEL C00000000015
07054820141228N201412220850D201412180300TH1400MO085000040300UL180001

Best How To :

There is no need to loop through the file with a bash loop. sed alone can handle it:

$ sed -r '/^.{14}R.{12}D/s/(.{52}).{4}/\10000/' file
05196220141228R201412241308D201412200055SA1155WE130800001300SL07Y051
05196220141228R201412241308A201412220350SA0731SU1950LAX C00020202020
05196220141228R201412241308D201412200055SA1155WE130800005300SL07Y051
05196220141228N201412241308A201412240007TU0548WE1107MEL C00000000015
07054820141228N201412220850D201412180300TH1400MO085000040300UL180001

This uses the expression sed '/pattern/s/X/Y/' file: in lines matching pattern, replace X with Y.

In this case,

/^.{14}R.{12}D/ line starts with any 14 characters followed by R, then any 12 characters followed by D.
(.{52}).{4} look for any 52 characters followed by any 4 characters and replace them with...
\10000 the first block followed by 0000.

SFTP Processbuilder

java,unix,sftp,processbuilder,ssh2-sftp

Two possible approaches: Use the scp command instead. It does the same ssh-based file transfer, but allows you to specify source and destination on the command line. ProcessBuilder pb = new ProcessBuilder("scp", "-i privateKey", "-r", "localFileDirectory", "[email protected]:remoteDirectory"); The -r is for "recursive", needed if you are transferring a whole folder....

How to remove characters before and including an underscore?

linux,string,bash,unix,awk

Using Parameter Expansion: $ var="fooo_barrrr" $ echo ${var#*_} barrrr To change the var itself, var=${var#*_}. Note this removes up to the first _: $ var="fooo_barrr_r" $ echo ${var#*_} barrr_r If you wanted to remove up to the last one, you would need to use ## instead: $ var="fooo_barrr_r" $ echo...

How to do the expansion of variable in shell script? [duplicate]

linux,bash,shell,unix,ksh

Using bash -c: newvar="$(bash -c "echo $var")" Using eval: newvar="$(eval "echo $var")" Example: #!/bin/bash var='$PATH' echo "$var" #This will show that $var contains the string $PATH literally #Using bash -c newvar="$(bash -c "echo "$var"")" echo "$newvar" #using eval newvar="$(eval "echo "$var"")" echo "$newvar" It will print the environment variable paths...

how to check a directory exist and can write file

oracle,shell,unix,sql-loader

For Checking Diretory is present if [ -d "$YOUR_DIRECTORY" ] then #Operation when directory is present else #Operation when directory is not present fi For checking if directory is writeable if [ -w "$YOUR_DIRECTORY"] then #Operation when directory is writeable else #Operation when directory is not writeable fi Regarding Oracle...

Multiple line search in a file using java or unix command

java,shell,unix,command

You can find like this. File file = new File("data/pattern.txt"); Pattern pat = Pattern.compile("subclass \"Pool1\" 11:22:33:44:55:66 \\{\\s*dynamic;\\s*\\}"); String content = Files.lines(file.toPath()).collect(Collectors.joining("\n")); Matcher m = pat.matcher(content); while (m.find()) { System.out.printf("found at %d-%d%n", m.start(), m.end()); } ...

Replace string :"{ using regex

.net,regex,replace

I'm pretty sure that you don't need a regular expression for this. Just use var str2 = str.Replace(":\"{", ":{").Replace("}\"", "}"); ...

conditional replace based off prior value in same column of pandas dataframe python

python,pandas,replace,fill,calculated-columns

Here's a kind of brute-force method. There is probably something more elegant, but you could explicitly loop over the rows like this: df = pd.DataFrame([0, -1, -1, -1, 0 , 0, 0, 1, 0]) df.columns = ['A'] df['B'] = df['A'] # loop here for i in range(1,len(df)): if df.A[i] ==...

Identifying when a file is changed- Bash

bash,shell,unix

I would store the output of find, and if non-empty, echo the line break: found=$(find . -name "${myarray[i]}") if [[ -n $found ]]; then { echo "$found"; echo " "; } >> "$tmp" fi ...

How to fix btrfs root inode errors

unix,fsck,btrfs

Provided that the broken inodes are the only problem present, the solution is to simply remove them. There may be a quicker way to do this, but here is what worked for me. From here I gleaned that you can use the find command to search for an inode like...

How to find average and maximum in an interval using Shell [closed]

linux,bash,shell,unix,awk

Please make a search before you ask any question many posts are already there You can try something like below, modify accordingly Input [[email protected] tmp]$ cat input.txt 1 3 2 5 3 4 4 3 5 2 6 1 7 3 8 3 9 4 10 2 11 2 12...

How to alter dynamic/live element attributes - (replaceWith()) used

jquery,dynamic,replace,live

I think the only thing you're maybe not wrapping your head around is the idea of callbacks and asynchronous methods. You're currently running your console.log statements before the replaceWith occurs. The function() { } block (your "callback") passed as the second parameter to $.get doesn't execute until the AJAX call...

What is wrong with these formats for Unix Shell Scripting Comparing Strings [duplicate]

bash,unix

Spaces in Bash are crucial not optional. Change the if statement to: if [ "$uInput" != "n" ]; then If you are wondering why? see this: Why should be there a space after '[' and before ']' in the Bash Script...

Use Unix Executable File to Run Shell Script and MPKG File

osx,shell,unix

The most common issue when handling variables containing paths of directories and files is the presence of special characters such as spaces. To handle those correctly, you should always quote the variables, using double quotes. Better code would therefor be: sudo sh "$path/join.sh" sudo sh "$path/join2.sh" It is also advised...

Batch-Script: Replace every “@” in file name with “_” in network drive including subfolders

batch-file,replace,folder,filenames

setsyntax for replacing a string: set "var1=my old hat" set "var2=%var1:old=new%" echo %var1% was better than %var2% To get all txt-files: for %%i in (*.txt) do ( echo %%i rem substitution does not work with special %%i variable type set "oldname=%%i" set "newname=!oldname:@=_!" echo rename "!oldname!" "!newname!" ) Why !...

javascript replace dot (not period) character

javascript,regex,replace

Try using the unicode character code, \u2022, instead: message.replace(/\u2022/, " \u2022"); ...

Regex to remove `.` from a sub-string enclosed in square brackets

c#,.net,regex,string,replace

To remove all the dots present inside the square brackets. Regex.Replace(str, @"\.(?=[^\[\]]*\])", ""); DEMO To remove dot or ?. Regex.Replace(str, @"[.?](?=[^\[\]]*\])", ""); ...

How to open a new terminal from my working terminal with same directory in Linux?

linux,unix,ssh

if you connect to your remote server via ssh -x [host] now run gnome-terminal & this will open a terminal with the same ssh connection. is this what your were after?...

Enumerating executable files in C

c,unix,directory,ansi

ep->d_name contains only relative pathname of the directory entry. So you should change the Current Working Directory to /tmp/hi before calling stat(2) if (chdir("/bin") != 0) { perror("chdir()"); exit(EXIT_FAILURE); } /* ... */ if (stat(ep->d_name, &sb) == -1) { perror("stat()"); exit(EXIT_FAILURE); } As noted in the comments by @Andrew Medico,...

Return value based on duplicate columns

mysql,sql,replace,duplicates

You can try with group by: select Firstname , Lastname , case when count(*) > 1 then '**ERROR**' else age end from Table group by Firstname , Lastname , age; Or in case you want to return all rows with duplicates: select t.Firstname , t.Lastname , case when (select count(*)...

Replacing elements in an HTML file with JSON objects

javascript,json,replace

obj.roles[0] is a object {"name":"with whom"}. you cant replace string with object. you need to refer to property "name" in the object obj.roles[0].name Another problem is that var finalXML get a new value every line. you need to add a new value to the variable, not replcae it. var finalXML...

Why can I view some Unix executable files in Mac OS X and not others?

git,bash,shell,unix,binary

Executable files may be scripts (in which case you can read the text), or binaries (which are ELF formatted machine code). Your shell script is a script; git is an ELF binary. You can use the file command to see more detail. For example, on my nearest Linux system: $...

Split a column's values into multiple rows awk/unix/python

python,unix,awk

This may help you $ cat file header: id,indicator,{(pid,days_remaining)} row: id_558314,1,{(property_66021,7),(property_24444,1),(property_285395,6)} $ awk -F, '{gsub(/[{}()]/,"")}FNR==1{print;next}{j=0;p=$1;for(i=3; i<=NF; i+=2){ $1=p;sub(/:/,++j"&",$1);print $1,$2,$i,$(i+1)}}' OFS=, file header: id,indicator,pid,days_remaining row1: id_558314,1,property_66021,7 row2: id_558314,1,property_24444,1 row3: id_558314,1,property_285395,6 Better Readable version awk -F, '{ gsub(/[{}()]/,"") } FNR==1{ print next } { j=0 p=$1 for(i=3;...

Mounting GEOM_ELI Encrypted ZFS Pool as root

unix,encryption,freebsd,boot,zfs

Turns out I was correct. The daXp4.eli files are necessary as it's the metadata of each disk. A reference point if you will. By performing: geli backup /dev/daXp4 /boot/daXp4.eli It create the meta files required for geom to attempt a decryption at boot time. I hope this helps someone else...

Selecting unique lines based on two columns

unix,awk

You can perfectly use an index that uses more than one field for the array elements: awk -F"\t" '!seen[$2, $3]++' file In this case we use $2, $3 as index. This way, we will get all different elements of the tuples ($2, $3)....

How to extract single-/multiline regex-matching items from an unpredictably formatted file and put each one in a single line into output file?

linux,shell,unix,replace,grep

Assuming that your document is well-formed, i.e. opening tags always match with a closing tag, then this may be what you need: sed '[email protected]<[/]\?b>@\n&\[email protected]' path/to/input.txt | awk 'BEGIN {buf=""} // {Y=1; buf=""} /<\/b>/ {Y=0; print buf""} Y {buf = buf$0} ' | tr -s ' ' Output: data1...

How to replace href inside of
using jQuery

jquery,replace,html-lists,href

$("li #ctl01_Auxiliary_Auxiliary_rptWrapper_Auxiliary_rptWrapper_rpt_ctl01_Navigat‌ion‌Link").attr("href", "/Public/Cart") ...

Converting values from for loop to json format

linux,unix

Observe that each line except the first begins with ",\n". dir="myfiles/test/" prefix="" echo "[" >> test.json for dir in "${array[@]}"; do #reverse the result of comparisons file=$(find "$dir" -maxdepth 1 -type f -iname '*.txt' | awk "NR==$i") [[ -n $file ]] && printf '%b{ "filepath": "%s" }' $prefix "$file" >>...

Python - Opening and changing large text files

python,replace,out-of-memory,large-files

You need to read one bite per iteration, analyze it and then write to another file or to sys.stdout. Try this code: mesh = open("file.mesh", "r") mesh_out = open("file-1.mesh", "w") c = mesh.read(1) if c: mesh_out.write("{") else: exit(0) while True: c = mesh.read(1) if c == "": break if c...

Using ant.replace in gradle

replace,ant,gradle

It should be: task writeVersion << { ant.replace( file: 'version.txt', token: 'versionNumber', value: '1.0.0' ) } and: version.number=versionNumber ...

shell script for counting replacements

bash,replace,count

Assuming you want to replace the word 'apple' with 'banana' (exact match) in the contents of the files and not on the names of the files (see my comment above) and that you are using the bash shell: #!/bin/bash COUNTER=0 for file in *.txt ; do COUNTER=$(grep -o "\<apple\>" $file...

Capture tee's argument inside piped Perl execution

perl,unix

The short answer is - you can't. tee is a separate process with it's own arguments. There is no way to access these arguments from that process. (well, I suppose you could run ps or something). The point of tee is to take STDOUT write some of it to a...

Redirect output from file to stdout

bash,shell,unix,stdout

Many tools interpret a - as stdin/stdout depending on the context of its usage. Though, this is not part of the shell and therefore depends on the program used. In your case the following could solve your problem: myprog -o - input_file ...

Looping Over a String Gives Segmentation Fault

c++,string,replace,lowercase

Turning on the warnings, you may spot the error yourself: g++ -o main main.cpp -Wall Errors: main.cpp: In function ‘std::string lowercase(std::string)’: main.cpp:9:21: warning: comparison between signed and unsigned integer expressions [-Wsign-compare] for (int i=0; i < ans.size(); i++) ^ main.cpp:14:1: warning: no return statement in function returning non-void [-Wreturn-type] }...

Extracting columns within a range AWK

unix,awk

You want to test for 0.75-0.8 but wrote code to test for 0.7-0.75 and you forgot to specify what to test in the second part of your condition. Do this: awk '$2 >= 0.75 && $2 <= 0.8' Also note that you want a numeric comparison not a string comparison...

find numbers divisible by 3 in csv file using shell script

bash,shell,unix,awk

awk -F'|' '{for(i=1;i<=NF;i++)if(!($i%3))print $i}' file this awk one-liner shoud do. With your example, the cmd outputs: 3 6 9 ...

Join Statement omitting entries

unix,join,hidden-characters

I tried this out, and I noticed a couple things. First: this is minor, but I think you're missing a comma in your -o specifier. I changed it to -o 1.1,2.1. But then, running it on just the fragments you posted, I got only three lines of output: 1|1 3|3...

Replace backslash in string

java,regex,string,exception,replace

Use four backslashes to match a single backslash character. String texter = texthb.replaceAll("\\\\.+?\\\\", "\\\\"+String.valueOf(pertotal + initper)+"\\\\"); ...

Search and replace all php_short_tag that is

php,regex,replace,php-shorttags,shorttags

Just make the = symbol as optional one. preg_replace('~<\?=?(?!php\b)~', '<?php ', $str); OR preg_replace('~<\?=?(?!php\b)(\w)~', '<?php \1', $str); DEMO...

pass enter key from Java to Shell script

java,unix,jsch

According to the JSch javadoc, you must call setInputStream() or getOutputStream() before connect(). You can only do one of these, once. For your purposes, getOutputStream() seems more appropriate. Once you have an OutputStream, you can wrap it in a PrintWriter to make sending commands easier. Similarly you can use channel.getInputStream()...

Using regex to extract multiple strings

regex,xml,string,replace

Just use parentheses instead of braces. Regex.Replace(str, @"<(jv:(?:FirstName|MiddleInitial|LastName)>).*?</\1, "<$1</$1"); Braces signify matching any character within them one time. Parentheses match the full string....

Calling find more than once on the same folder tree

linux,bash,shell,unix,find

Try this: find . -mmin +35 -or -mmin -25 find supports several logical operators (-and, -or, -not). See the OPERATORS section of the man pages for more details. ==================== EDIT: In response to the question about processing the two matches differently, I do not know of a way to do...

Text replacement from a string in php

php,arrays,regex,string,replace

That's a simple search and replace for your defined markers. $text = 'Hello World [this-echo] Hello Everybody [nobody]'; $markers=array('[this-echo]','[nobody]'); $replacements=array('Today','Anybody'); $text= str_replace($markers,$replacements,$text); Output Hello World Today Hello Everybody Anybody Fiddle...

storing 'du' result in a variable [duplicate]

bash,unix,putty

Like so FOO="$(du -m myfile.csv)" echo "$FOO" Output 1.25 myfile.csv ...

Unable to connect to mysql after automating mysql-server installation in bash script

mysql,linux,bash,unix

I think Álvaro its right about the quotes, try something like debconf-set-selections <<< "mysql-server mysql-server/root_password password $passdb" debconf-set-selections <<< "mysql-server mysql-server/root_password_again password $passdb" ...

How to do replace multiple times ? php

php,string,replace,str-replace,php-5.3

You can't achieve this by str_replace. Use string translation (strtr) which was designed to do so instead: $words = 'word1 word2'; $wordsReplaced = strtr($words, [ 'word1' => 'word2', 'word2' => 'word1' ]); ...

How to replace newlines/linebreaks with a single space, but only if they are inside of start/end regexes?

regex,linux,shell,unix,replace

You can do it using awk as: awk '/\[x/{f=1} {if(f)printf "%s",$0; else print $0;} /y\]/{print ""; f=0}' Output: [x data1 data2 data3 data4 y] [a data5 data 6 data7 data 8 b> [x data y] You can also simplify to: awk '/\[x/,/y\]/{ORS=""; if(/y\]/) ORS="\n";}{print}' Output: [x data1 data2 data3 data4...

Swift - how to convert hyphen/minus sign into + sign

string,swift,replace,converter

tmpAddress = tmpAddress.stringByReplacingOccurrencesOfString("-", withString: "+") This replaces '-' with '+' The first parameter is the search string, and the second is the replacement string....

Reading rsync source from file results in improper parsing of file names with white space

bash,shell,unix,scripting,rsync

Replace for JOB in `cat /tmp/jobs.txt`; do rsync -avvuh "$JOB" "$DESTINATION"; done by while read -r JOB; do rsync -avvuh "$JOB" "$DESTINATION" done < /tmp/jobs.txt ...