Expand one column while preserving another

I am trying to get column one repeated for every value in column two which needs to be on a new line.

cat ToExpand.txt

Pete    horse;cat;dog
Claire  car
John    house;garden

My first attempt:

cat expand.awk

BEGIN {
        FS="\t"
        RS=";"
}
{
        print $1 "\t" $2
}

awk -f expand.awk ToExpand.txt

Pete    horse
cat
dog
Claire  car
John
garden

The desired output is:

Pete    horse
Pete    cat
Pete    dog
Claire  car
John    house
John    garden

Am I on the right track here or would you use another approach? Thanks in advance.

Best How To :

You could also change the FS value into a regex and do something like this:

awk -F"\t|;" -v OFS="\t" '{for(i=2;i<=NF;i++) print $1, $i}' ToExpand.txt
Pete    horse
Pete    cat
Pete    dog
Claire  car
John    house
John    garden

I'm assuming that:

The first tab is the delimiter for the name
There's only one tab delimiter - If tab delimited data occurs after the ; section use fedorqui's implementation.

It's using an alternate form of setting the OFS value ( using the -v flag ) and loops over the fields after the first to print the expected output.

You can think of RS in your example as making "lines" out of your data ( records really ) and your print block is acting on those "lines"(records) instead of the normal newline. Then each record is further parsed by your FS. That's why you get the output from your first attempt. You can explore that by printing out the value of NF in your example.

AWK write to new column base on if else of other column

linux,bash,shell,awk,sed

You can use: awk -F, 'NR>1 {$0 = $0 FS (($4 >= 0.7) ? 1 : 0)} 1' test_file.csv ...

jQuery text animation from right to left

jquery,animation,text

Just add a height to ul ul.social-media-channels { margin: 0; padding:0; position:relative; float:right; margin-right:15px; display:none; height:20px; li{ display:inline; } } } http://codepen.io/anon/pen/JdyQRG...

Convert AWK command to sqlite query

sql,awk,sqlite3

SQLite is an embedded database, i.e., it is designed to be used together with a 'real' programming language. It might be possible to import that log file into a database file, but the whole point of having a database is to store the data, which is neither a direct goal...

Finding columns with only white space in a text file and replace them with a unique separator

regex,r,bash,awk,sed

$ cat tst.awk BEGIN{ FS=OFS=""; ARGV[ARGC]=ARGV[ARGC-1]; ARGC++ } NR==FNR { for (i=1;i<=NF;i++) { if ($i == " ") { space[i] } else { nonSpace[i] } } next } FNR==1 { for (i in nonSpace) { delete space[i] } } { for (i in space) { $i = "," } gsub(/,+/,",")...

Print the last 1,2,3..Nth or first 1,2,3…Nth matching block pattern using awk or sed

bash,awk,sed

This might work for you (GNU sed): sed -n '/pattern1/,/pattern2/{p;/pattern2/{H;x;s///2;x;T;q}}' file This prints the first 2 matches of pattern1 through pattern2 and then quits. sed -nr '/pattern1/,/pattern2/H;$!b;x;s/.*((pattern1.*){2})$/\1/p' file This prints the last 2 matches of pattern1 through pattern2....

AWK count number of times a term appear with respect to other columns

linux,shell,command-line,awk,sed

Almost same as the other answer, but printing 0 instead of blank. AMD$ awk -F, 'NR>1{a[$2]+=$3;b[$2]++} END{for(i in a)print i, a[i], b[i]}' File pear 1 1 apple 2 3 orange 0 1 peach 0 1 Taking , as field seperator. For all lines except the first, update array a. i.e...

Python outputs a .txt file, which's format differs depending on the text editor I use to open it

python,text,output

That is a problem with notepad itself. It can't handle "Linux newlines" instead it only recognizes "windows newlinew", so you have to write \r\n and then you will see the linebreaks in notepad.

Taking multiple header (rows matching condition) and convert into a column

bash,perl,command-line,awk,sed

In awk awk -F, 'NF==1{a=$0;next}{print a","$0}' file Checks if the number of fields is 1, if it is it sets a variable to that and skips the next block. For each line that doesn't have 1 field, it prints the saved variable and the line And in sed sed -n...

Remove part of a column, if its in a specific Column number. (The column has a variable)

bash,awk,sed

Re-using your awk variable definitions: $ awk -v old="Reading Comprehension " -v new="" -v col=6 'BEGIN{FS=OFS=","} {sub(old,new,$col)} 1' file Last,First,A00XXXXXX,1492-01-10,2015-06-17,,Sentence Skills 104,,Elementary Algebra 38, Last,First,A00XXXXXX,1492-01-10,2015-06-17,,,,Elementary Algebra 101,College Level Math 56 Last,First,A00XXXXXX,1492-01-10,2015-06-17,102,,,, Last,First,A00XXXXXX,1492-01-10,2015-06-17,,,,Elementary Algebra 118,College Level Math 97 Get the book Effective Awk Programming, 4th Edition, by Arnold Robbins....

Suppressing system command called from awk script

windows,awk,system

I have no idea what the magical Windows incantations are to control what displays on the terminal but in general instead of calling system() and letting the command it calls produce it's own output that's getting mixed in with the awk output, use getline to read the result of the...

Batch - Comparing two txt files

windows,batch-file,text,comparison

The code below keep lines from File2.txt from the first line that differ vs. File1.txt on: @echo off setlocal EnableDelayedExpansion rem Redirect the *larger* file as input < File2.txt ( rem Merge it with lines from shorter file for /F "delims=" %%a in (file1.txt) do ( rem Read the next...

loop add a comma after nth comma using awk

bash,awk

In my opinion it can be a difficult task to solve it with awk. Here you have an approach with perl. The difference is that I can use a hash, sort it and add items in the middle of an array with splice. perl -F',' -lanE ' BEGIN { %h...

How to match and change strings in a column of a semicolon separated file?

regex,awk,sed,gawk

There's no need for a separate condition with gsub - you can just apply it to each record and it won't do anything for those that don't match: awk -F\; -v OFS=";" '{gsub(/value/,"column 8",$8)}1' infile.csv > outfile.csv It is very important that you escape/quote the ; so that it isn't...

search string in textfile [on hold]

c++,c,string,text

If using C/C++ is not mandatory, then grep is probably your friend : grep "size [0-9]*" -o yourfile.txt > all_sizes.txt And if you only need the 50 first results, head it is : head -n 50 all_sizes > result.txt (Now, this assumes you're using some kind of Unix, or OS...

jQuery Make Text Change Before It's Noticeable

javascript,jquery,html,dom,text

You can display the anchor as hidden initially with css. The update your text and show with jquery. jQuery(document).ready(function() { jQuery('div#sidebar ul.side-nav li.current_page_item a').text('Request a Quote').show(); }); div#sidebar ul.side-nav li.current_page_item a { display: none; } ...

Finding the average of a column excluding certain rows using AWK

linux,bash,awk,scripting

Through awk, $ awk '$5!="99999"{sum+=$5}END{print sum}' file 227.5 Explanation: $5!="99999" if 5th column does not contain 99999, then do {sum+=$5} adding the value of 5th column to the variable sum. Likewise it keeps adding the value of 5th column when awk see's the record which satisfies the given condition. Finally...

Split a column's values into multiple rows awk/unix/python

python,unix,awk

This may help you $ cat file header: id,indicator,{(pid,days_remaining)} row: id_558314,1,{(property_66021,7),(property_24444,1),(property_285395,6)} $ awk -F, '{gsub(/[{}()]/,"")}FNR==1{print;next}{j=0;p=$1;for(i=3; i<=NF; i+=2){ $1=p;sub(/:/,++j"&",$1);print $1,$2,$i,$(i+1)}}' OFS=, file header: id,indicator,pid,days_remaining row1: id_558314,1,property_66021,7 row2: id_558314,1,property_24444,1 row3: id_558314,1,property_285395,6 Better Readable version awk -F, '{ gsub(/[{}()]/,"") } FNR==1{ print next } { j=0 p=$1 for(i=3;...

Using a command-line utility to perform the following map-updates

shell,command-line,awk,terminal

If order is not important, join and awk can do the job easily. $ join <(sort input.txt) <(sort mapping.txt) | awk -v OFS="|" '{for (i=3;i<NF;i++) print $2, $i OFS}' 103823054|001| 103823044|011| 103823044|012| 103823044|013| 103823064|011| 103823064|012| 103823064|013| ...

Using AWK to parse fields with commas

regex,bash,shell,awk

I'm not willing to wade through your whole question (sorry, IMHO it's just too long with too much extraneous information) but it looks like you're trying to extract the individual fields from that "confile1" at the top of your question so maybe this is all the hint you need: $...

Using blank-line delimited records and colon-separated fields in awk

awk,etl

Method 1 We use split to split each field into two parts: the key and the value. From these, we create associative array a: $ awk -F'\n' -v RS= '{for (i=1;i<=NF;i++) {split($i,arr,/: /); a[arr[1]]=arr[2];} if (a["Age"]+0>40) print a["Name"];}' file Smith, John Mills, Pat Method 2 Here, we split fields at...

Removing the blank positions (x,y) from a text file in Python?

python,text

If this is a simple text file and each frame is in a new line. You can simply create a new file with the frames where there is only 1 element in the line and add 12 nan to those lines. Example code - with open('input.txt','r') as f , open('newoutput.txt','w')...

Extracting Substring from File Name

string,awk,substring,extract,cut

I would use sed: $ echo "asdfasd_d20150616asdasd" | sed -r 's/^.*_d(.{8}).*$/\1/' 20150616 This gets a string and removes everything up to _d. Then, catches the following 8 characters and prints them back. sed -r is used to be able to catch groups with just () instead of . ^.*_d(.{8}).*$ ^...

awk ternay operator, count fs with ,

awk

awk -F, '{ print ( NF ? $NF : $0 ) }' file ...

find numbers divisible by 3 in csv file using shell script

bash,shell,unix,awk

awk -F'|' '{for(i=1;i<=NF;i++)if(!($i%3))print $i}' file this awk one-liner shoud do. With your example, the cmd outputs: 3 6 9 ...

Check for decimal point and add it at the end if its not there using awk/perl

regex,perl,shell,awk

In awk Just sub for those fields awk -F, -vOFS="," '{sub(/^[^\.]+$/,"&.",$6);sub(/^[^\.]+$/,"&.",$11)}1' file or sed sed 's/^$\([^,]*,$\{5\}[^.,]\+\),/\1./;s/^$\([^,]*,$\{10\}[^.,]\+\),/\1./' file ...

Extracting columns within a range AWK

unix,awk

You want to test for 0.75-0.8 but wrote code to test for 0.7-0.75 and you forgot to specify what to test in the second part of your condition. Do this: awk '$2 >= 0.75 && $2 <= 0.8' Also note that you want a numeric comparison not a string comparison...

Replace [a-z],[a-z] with [a-z], [a-z] and keep the letters

bash,awk,sed

What I have tried sed 's/[a-z],[a-z]/[a-z], [a-z]/g' <<< "suny stony brook, stony brook,usa." You need to use regex's capture groups here to refer to the original [a-z] values. For example: s/$[a-z]$,$[a-z]$/\1, \2/g Notice how I've surrounded those [a-z] with $ and $? These form capture groups that can be...

c++ gives Segmentation Fault from text file of all 0's

c++,text,zero,fault

For the second while loop, did you want to have: while ( Min >> value ) { M.push_back(value); } You're getting the segmentation fault because the Ein has hit EOF. Also, you may want to close the input streams :)...

if a string exist (including a variable), flip it using awk or sed

bash,awk,sed

Using sed with extended regex: sed -r 's/(College Level Math.*?),(Elementary Algebra.*)/\2,\1/g' filepath \1 is the first captured group (College Level Math.*?) \2 is the second captured group (Elementary Algebra.*) Thus the above sed performs a replacement operation from \1,\2 to \2,\1 The above regex will serve for general cases, but...

Text showing on two lines with span

html,css,text

Add white-space:nowrap to the span: span.navbar-brand{ white-space:nowrap; width:auto; } http://jsfiddle.net/kk60youo/...

How to split a CSV file into multiple files based on column value

bash,csv,awk

You can use awk to generate a file containing only a particular value of the second column: awk -F ';' '($2==1){print}' data.dat > data1.dat Just change the value in the $2== condition. Or, if you want to do this automatically, just use: awk -F ';' '{print > "data"$2".dat"}' data.dat which...

How to find average and maximum in an interval using Shell [closed]

linux,bash,shell,unix,awk

Please make a search before you ask any question many posts are already there You can try something like below, modify accordingly Input [[email protected] tmp]$ cat input.txt 1 3 2 5 3 4 4 3 5 2 6 1 7 3 8 3 9 4 10 2 11 2 12...

How to append entry the end of a multi-line entry using any of stream editors like sed or awk

linux,bash,awk,sed,sh

Here's a sed version: /^Host_Alias/{ # whenever we match Host_Alias at line start : /\\$/{N;b} # if backslash, append next line and repeat s/$/,host25/ # add the new host to end of line } If you need to add your new host to just one of the host aliases, adjust...

Change a Script to a For Do Done Loop

linux,bash,for-loop,awk

Turns out the code wasn't invalid (had to correct some quoting issues) but that the folder was corrupt when i tried to use it in the bash script. Here is the working code with the correct double quotes around the directory variables. #!/bin/bash #file location XMLDIR='/home/amoore19/XML/00581-001/scores' NEWXML='/home/amoore19/XML/00581-001' #this gives me...

Access a bash array in awk loop

arrays,bash,awk

To my understanding, you just need to feed awk with the bash array in a correct way. That is, by using split(): awk -v bash_array="${myarray[*]}" 'BEGIN{split(bash_array,array); FS=OFS="\t"} NR>=1{for (i=1;i<=NF;i++) a[i]+=$i} END{for (i=1;i<NF;i++) print a[i], array[i]}' file Since the array array[] is now in awk, you don't have to care about...

How to remove characters before and including an underscore?

linux,string,bash,unix,awk

Using Parameter Expansion: $ var="fooo_barrrr" $ echo ${var#*_} barrrr To change the var itself, var=${var#*_}. Note this removes up to the first _: $ var="fooo_barrr_r" $ echo ${var#*_} barrr_r If you wanted to remove up to the last one, you would need to use ## instead: $ var="fooo_barrr_r" $ echo...

Can I put StreamReaders in a list? Or any other way to read a lot of text files at once?

c#,list,text,streamreader

You are not using curly braces, so you cannot see where the object is disposed. You code is identical to this code: List<StreamReader> lijst = new List<StreamReader>(); using (StreamReader qwe = new StreamReader("C:\\123.txt")) { using (StreamReader qwer = new StreamReader("C:\\1234.txt")) { lijst.Add(qwe); } } lijst.Add(qwer); This means that when you...

How can I wrap text in QGraphicsItem?

qt,text,word-wrap,qgraphicsitem

You did not specify Qt version but try: void QGraphicsTextItem::setTextWidth(qreal width) Sets the preferred width for the item's text. If the actual text is wider than >the specified width then it will be broken into multiple lines. If width is set to -1 then the text will not be broken...

Repeating the format specifiers in awk

awk,printf,gawk

This may help you [[email protected] tmp]$ cat test.for implicit none integer i write(*,'(10I5)')(i,i=1,100) end [[email protected] tmp]$ gfortran test.for [[email protected] tmp]$ ./a.out 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29...

Python - Select specific value from test file

python,file,text

You could split the text and have a list of lists, where each sub list is a row, then pluck whatever you need from the list using rows[row - 1][column - 1]. f = open('test.txt', 'r') lines = f.readlines() f.close() rows = [] for line in lines: rows.append(line.split(' ')) print...

Bash modify CSV to change a field

linux,bash,awk

Please save following awk script as awk.src: function date_str(val) { Y = substr(val,0,4); M = substr(val,5,2); D = substr(val,7,2); date = sprintf("%s-%s-%s",Y,M,D); return date; } function time_str(val) { h = substr(val,9,2); m = substr(val,11,2); s = substr(val,13,2); time = sprintf("%s:%s:%s",h,m,s); return time; } BEGIN { FS="|" } # ## MAIN...

Powershell Reading text file

powershell,text,text-files

To read the text after the # characters you must read the file content up to the # characters first. Also, in PowerShell you normally read files either line by line (via Get-Content) or completely (via Get-Content -Raw). You can discard thos parts of the read content that don't interest...

how to deletes line from a text file that are taken from another file [duplicate]

shell,awk,sed,grep,sh

Something like this with grep: grep -vxf lines.txt data.txt > no_dupplicate_lines.txt Sample: AMD$ cat lines.txt Line2 Line4 AMD$ cat data.txt Line1 Line2 Line3 Line4 Line5 AMD$ grep -vxf lines.txt data.txt Line1 Line3 Line5 Print the lines that are not matching (-v) the exact lines (-x) from the file lines.txt (-f...

Ignore first few lines and last few lines in a file Linux

linux,awk

awk cannot look ahead so you'll have to save the lines. awk 'NR>2{if(z!="")print z;z=y;y=x;x=$0}' file Practically zero memory overhead...

Trying to get just the text from a node with filter(), but is returning an object?

javascript,jquery,dom,text

You have a TextNode object so need to read it thusly: var text = $(textIWant).text(); Or natively var text = textIWant.nodeValue; ...

Bash, Using grep, sed or awk to extract section of text and then match

bash,awk,sed,grep

Another sed solution Will work for multiple blues sed -n '/^int/{x;/blue/{p;d}};/blue/H' file Input random text random text random text random text random text int 1 random text blue blue random text random text int 2 random text random text red random text int 3 random text random text random text...

Selecting unique lines based on two columns

unix,awk

You can perfectly use an index that uses more than one field for the array elements: awk -F"\t" '!seen[$2, $3]++' file In this case we use $2, $3 as index. This way, we will get all different elements of the tuples ($2, $3)....

Text justification C language

c,text,alignment

From printf's manual: The field width An optional decimal digit string (with nonzero first digit) specifying a minimum field width. If the converted value has fewer characters than the field width, it will be padded with spaces on the left (or right, if the left-adjustment flag has been given). Instead...

BASH - conditional sum of columns and rows in csv file

linux,bash,csv,awk

This awk program will print the modified header and modify the output to contain the sums and their division: awk 'BEGIN {FS=OFS=";"} (NR==1) {$10="results/time"; print $0} (NR>1 && NF) {sum8[$10]+=$8; sum9[$10]+=$9; other[$10]=$0} END {for (i in sum8) {$0=other[i]; $8=sum8[i]; $9=sum9[i]; $10=(sum9[i]?sum8[i]/sum9[i]:"NaN"); print}}' which gives: Date;dbms;type;description;W;D;S;results;time;results/time Mon Jun 15 14:22:20 CEST...

AWK|BASH, use double FS and ternary operator

bash,awk

You can use this awk command awk -F, '{if (split($1, a, " ") > 2) print $NF; else print}' file This will output rosario escuela estadist, medellin medellin If you want to get rid of the space before $NF, use this awk command awk -F, '{if (split($1, a, " ")...