Menu
  • HOME
  • TAGS

how to process big file with comparison of each line in that file with remaining all lines in same file?

string-comparison,opencsv,file-processing

It is taking a long time because it looks like you are reading the file a huge amount of times. You first read the file into the lines List, then for every entry you read it again, then inside that you read it again!. Instead of doing this, read the...

Unix - show some characters in file by line number

unix,sed,file-processing

You better use awk: awk 'NR==line_number {print substr($0,start_position,num_of_characters_to_show)}' file For example, print 5 characters starting from the 2nd character in the line 2: $ cat a 1234567890 abcdefghij $ awk 'NR==2 {print substr($0,2,5)}' a bcdef If you really need to use sed, you can use something like: $ sed -rn...

remove line from file if more than one pattern appears in different line

shell,awk,sed,file-processing

try: sort -k 6 f1.txt | uniq -f 5 assuming the original order of the lines doesn't matters....

PHP : how to process the files _alphabetically_ using DirectoryIterator?

php,file,directory,alphabetical,file-processing

DirectoryIterator objects provide a simple way to access many of file properties. $dir = new DirectoryIterator($path); foreach ($dir as $fileInfo) { if ((!$fileInfo->isDot())&&($fileInfo->GetExtension() == "txt")) { /* You can access the file information inside this cycle */ $octal_perms = substr(sprintf('%o', $fileInfo->getPerms()), -4); echo $fileInfo->getFilename() . " " . $octal_perms ....

delete lines from file in python

python,file-io,file-processing

We can use the next() function to get the next element in the file iterable. The shutil module allows us to move the new file, overwriting the original (thanks @JoranBeasley). import shutil with open(filePath, 'r') as f, open('new_' + filePath, 'w') as output: for line in f: n = next(f)...

python: text file processing, for loop and read operation conflict

python,for-loop,file-processing

What is happening here is the read operation returning the full contents of the file (thus placing the caret at the end of the file) by the time when you assign your variable, that is why you are receiving empty string. You need either do this: fhand2 = open('mbox-short.txt') inp...