SuperCSV with Dozer - How to map to a List of Lists?

Tag: csv,dozer,supercsv

Using SuperCSV with Dozer (version 2.2.0), I've been able to successfully map data into a List. However, I now have a case where I want to map into a List of Lists. In the object I'm trying to map to, I have a field that looks something like this:

List<List<String>> myListOfLists;

For my field mapping I did this:

"myListOfLists[0][0]", "myListOfLists[0][1]", "myListOfLists[1][0]", etc.

However, this results in the following error:

org.dozer.MappingException: No read or write method found for field (myListOfLists[0]) in class (class com.foo.MyClassBeingMappedTo)

I can't seem to find any examples of nested collections, but it seems like this should be possible. Is there a way I can map the data into my List<List<String>> field?

Best How To :

I got around this by creating a wrapper class for my inner List. I created an inner class like this:

public static class InnerListWrapper
{
    private List<String> innerList;

    public List<String> getInnerList()
    {
        return innerList;
    }
    public void setInnerList(List<String> innerList)
    {
        this.innerList = innerList;
    }
}

Then my outer List looks like this:

List<InnerListWrapper> myListOfLists;

Then the fields mapping is simply "myListOfLists[0].innerList[0]","myListOfLists[0].innerList[1]", ""myListOfLists[1].innerList[0]", etc.

Not the cleanest, but like the comment above says, Dozer doesn't seem to support nested Lists. So you have to add another class in between.

Compare 2 seperate csv files and write difference to a new csv file - Python 2.7

python,python-2.7,csv,compare

What do you mean by difference? The answer to that gives you two distinct possibilities. If a row is considered same when all columns are same, then you can get your answer via the following code: import csv f1 = open ("olddata/file1.csv") oldFile1 = csv.reader(f1) oldList1 = [] for row...

type conversion performance optimizable?

c#,xml,csv,optimization,type-conversion

IEnumerable<string> values = new List<string>(); values = … Probably not going to be a big deal, but why create a new List<string>() just to throw it away. Replace this with either: IEnumerable<string> values; values = … If you need values defined in a previous scope, or else just: Enumerable<string> values...

Perl: Using Text::CSV to print AoH

arrays,perl,csv

Pretty fundamentally - CSV is an array based data structure - it's a vaguely enhanced version of join. But the thing you need for this job is print_hr from Text::CSV. First you need to set your header order: $csv->column_names (@names); # Set column names for getline_hr () Then you can...

Python CSV reader/writer handling quotes: How can I wrap row fields in quotes? (Getting triple quotes as output)

python,csv

quotechar only indicates what character the writer should use for quoting. It's quote=csv.QUOTE_ALL that you need. Create your writer like this: a = csv.writer(fp, quoting=csv.QUOTE_ALL) quoting defaults to csv.QUOTE_MINIMAL, meaning that it will only quote fields if they contain the delimiter, which is why it's only quoting "JOHNSON, JOHN J."....

Converting list to array with NumPy asarray method

python,csv,python-3.x,numpy

Try this: from pandas import read_csv data = read_csv('country.csv') print(data.iloc[:,1].mean()) This code will convert your csv to pandas dataframe with automatic type conversion and print mean of the second column. ...

Create an external Hive table from an existing external table

csv,hadoop,hive

I am presuming you want to select distinct data from "uncleaned" table and insert into "cleaned" table. CREATE EXTERNAL TABLE `uncleaned`( `a` int, `b` string, `c` string, `d` string, `e` bigint ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' LOCATION '/external/uncleaned' create another...

Saying there are 0 arguments when I have 2? Trying to open a CSV file to write into

ruby,file,csv,dir

I believe the problem is with Dir.foreach, not CSV.open. You need to supply a directory to foreach as an argument. That's why you are getting the missing argument error. Try: Dir.foreach('/path/to_my/directory') do |current_file| I think the open that is referenced in the error message is when Dir is trying to...

Compare 2 csv files and output different rows to a 3rd CSV file using Python 2.7

python,python-2.7,csv,compare

You're currently checking for rows that exist in the old file but aren't in the new file. That's not what you want to do. Instead, you should check for rows that exist in the the new file, but aren't in the new one: output = [row for row in newList2...

CSV File header part is coming parsing by LINQ

c#,linq,csv

You seem to be doing the Skip(1) in the wrong place: var csvLinesData = csvlines.Skip(1).Select(l => l.Split(',').ToArray()); // IEnumerable<string[]> As it stands you're skipping the first column for each row, not the first row....

export mysql query array using fputcsv

php,mysql,arrays,csv,fputcsv

Corresponding to official manual for mysqli_fetch_array: mysqli_fetch_array — Fetch a result row as an associative, a numeric array, or both You coded MYSQLI_ASSOC flag, so yo get associative array for one row of data: By using the MYSQLI_ASSOC constant this function will behave identically to the mysqli_fetch_assoc() See examples of...

Group instances based on NA values in r

r,file,csv,instance,na

df[!is.na(df$Value), ] Size Value Location Num1 Num2 Rent 1 800 900 <NA> 2 2 y 3 1100 1300 uptown 3 3 n 4 1200 1100 <NA> 2 1 y And df[is.na(df$Value), ] Size Value Location Num1 Num2 Rent 2 850 NA midcity NA 3 y 5 1000 NA Lakeview NA...

Adding time/duration from CSV file

python,python-2.7,csv,datetime

Use datetime.timedelta() objects to model the durations, and pass in the 3 components as seconds, minutes and hours. Parse your file with the csv module; no point in re-inventing the character-separated-values-parsing wheel here. Use a dictionary to track In and Out values per user; using a collections.defaultdict() object will make...

How to rearrange CSV / JSON keys columns? (Javascript)

javascript,json,csv,papaparse

Papa Parse allows to specify order of fields in the unparse() function: var csv = Papa.unparse({ fields: ["ID", "OrderNumber", "OrderStatus", "StartTime", "FinishTime", "canOp", "OpDesc", "UOM"], data: [{ OrderStatus: "Good", canOp: "True", OpDesc: "Good to go", ID: "100", OrderNumber: "1000101", FinishTime: "20:50", UOM: "K", StartTime: "18:10" }, // ... ] });...

Exporting Data from Cassandra to CSV file

apache,csv,cassandra,export,export-to-csv

The syntax of your original COPY command is also fine. The problem is with your column named timestamp, which is a data type and is a reserved word in this context. For this reason you need to escape your column name as follows: COPY product (uid, productcount, term, "timestamp") TO...

How to find the difference between two lists of dictionaries checking the key-value pair

python,list,csv,dictionary

Part of this solution was found by OP as per our last CHAT conversation, it was to convert a string into dictionary using ast module. Now using this module to convert every row read by the csv.reader() as it returns a list of strings, which would be a list of...

What would be the fastest way to insert this data

mysql,sql,database,performance,csv

You can import the CSV file into a separate table using mysql LOAD DATA INFILE and then update the entries table using JOIN statement on the basis of similar column name. E.g: update entries a inner join new_table b on a.name = b.name set a.address = b.address ; Here new_table...

Converting very large files from xml to csv

c#,xml,csv,converter,large-files

You need to take a streaming approach, as you're currently reading the entire 2Gb file into memory and then processing it. You should read a bit of XML, write a bit of CSV and keep doing that until you've processed it all. A possible solution is below: using (var writer...

Hive external table not reading entirety of string from CSV source

csv,hadoop,hive,hiveql

What version of Hive are you using? On Amazon EMR, Hive version 0.13.1 I run your code and get the following hive> CREATE EXTERNAL TABLE BG ( > `Id` string, > `Someint` int > ) > ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' > LOCATION '/tmp/example' > TBLPROPERTIES ("skip.header.line.count"="1"); OK...

How to stop foreach loop from printing duplicate data?

php,csv,foreach

As your foreach is printing duplicates , it means that your array $csv_tbl contains duplicate values, you can remove duplicate values from array using array_unqiue But also by looking at the screenshots i can see that the callid is different for every record. do your csv contain duplicates?:: $k =1;...

BASH - conditional sum of columns and rows in csv file

linux,bash,csv,awk

This awk program will print the modified header and modify the output to contain the sums and their division: awk 'BEGIN {FS=OFS=";"} (NR==1) {$10="results/time"; print $0} (NR>1 && NF) {sum8[$10]+=$8; sum9[$10]+=$9; other[$10]=$0} END {for (i in sum8) {$0=other[i]; $8=sum8[i]; $9=sum9[i]; $10=(sum9[i]?sum8[i]/sum9[i]:"NaN"); print}}' which gives: Date;dbms;type;description;W;D;S;results;time;results/time Mon Jun 15 14:22:20 CEST...

Python: isolating re.search results

python,regex,csv

The problem here is that re.search returns a match object not the match string and you need to use group attribute to access your desire result. If you wants all the captured groups you can use groups attribute and for a special group you can pass the number of expected...

Entity framework to CSV float values

c#,entity-framework,csv

Probably in your case the simplest way to do it is to change the current thread culture then restore it. Thread.CurrentThread.CurrentCulture = new CultureInfo("en-US"); You can also specify a format provider in (prop.GetValue(entityObject, null) ?? "?").ToString() but in this case you probably need to check if prop.GetValue(entityObject, null) is IFormattable...

Is it possible to output to a csv file with multiple sheets?

java,excel,csv

CSV file is interpreted a sequence of characters which comply to some standardization, therefor it cannot contains more than one sheet. You can output your data in a Excel file that contains more than one sheet using the Apache POI api.

Python: Using panda to import csv. Trying to plot a column but gives me an error saying “no numerical data to plot”

python,csv,pandas,plot

The problem is, since each of your columns has a non-numeric value in the first non-header row, pandas automatically parses the entire column to be text. One workaround is to skip the text row like this: df=pd.read_csv(filepath, sep=",",skiprows=[1]) Then when you try to plot it will work just using: df['coal...

Extract Values from CSV file to then multiply with values from other CSV? Python

python,csv

Here's one way that demonstrates using csv. It's repetitive and verbose, in hopes it'll give you something to research and test with. #!/usr/bin/python import csv import re file2_list = [] with open('2.csv', 'r') as fh2: reader = csv.reader(fh2) for row in reader: if re.search(r'\d', row[0]): file2_list = row break with...

Read CSV and plot colored line graph

python,csv,matplotlib,graph,plot

you need to turn x and y into type np.array before you calculate above_threshold and below_threshold, and then it works. In your version, you don't get an array of bools, but just False and True. I added comma delimiters to your input csv file to make it work (I assume...

How Do I Transform This CSV / Tabular Data Into A Different Shape?

excel,csv

Assuming your events are in different columns, something like this would work - Sub test() Dim wsOrig As Worksheet Set wsOrig = ActiveSheet Dim wsDest As Worksheet Set wsDest = Sheets("Sheet2") Dim lcol As Integer lcol = Cells(1, Columns.Count).End(xlToLeft).Column Dim lrow As Integer lrow = Cells(Rows.Count, "A").End(xlUp).row Dim x As...

How to collect data from text file to dict in Python?

python,python-2.7,csv,dictionary

I think I understand the table that you have, but if the following does not work let me know. I have tried to make this code as generic as possible (i.e. reading in the header line and not assuming 4 bases as header so this could work for say a...

Run 3 variables at once in a python for loop.

python,loops,variables,csv,for-loop

zip the lists and use a for loop: def downloadData(n,i,d): for name, id, data in zip(n,i,d): URL = "http://www.website.com/data/{}".format(name) #downloads the file from the website. The last part of the URL is the name r = requests.get(URL) with open("data/{}_{}_{}.csv".format(name, id, data), "wb") as code: #create the file in the format...

How to process the irregular json file in python to get the serialized values in CSV?

python,json,csv

Python 2.7.8 (default, Oct 20 2014, 15:05:19) >>> data = """{\n \"1 & 1 Internet\": { ... \n \"category\": \"Infrastructure\", ... \n \"xyz\": 55, ... \n \"ABC\": \"low\" ... \n }, ... \n \"1 website hosting\": { ... \n \"category\": \"Infrastructure\", ... \n \"xyz\": 0, ... \n \"ABC\": \"poor\" ......

Adding data from a csv file to a List using a loop

c#,list,csv

mySales is a class, which is a reference type. In option 2, you are doing the following: You create mySale, which now is a reference to a specific instance of mySales. In the for loop, you modify mySale (which is still a reference to that exact same instance), and then...

Convert delimited string to array and group using LINQ in C#

c#,arrays,linq,csv

public static void Main() { var input = @"**Albert School**: George Branson, Eric Towson, Nancy Vanderburg; **Hallowed Halls**: Ann Crabtree, Jane Goodall, Rick Grey, Tammy Hudson; **XXX University**: Rick Anderson, Martha Zander;"; var universities = input .Split(';') .Select(ParseUniversity) .ToArray(); } public static University ParseUniversity(string line) { var split = line...

How to split a CSV file into multiple files based on column value

bash,csv,awk

You can use awk to generate a file containing only a particular value of the second column: awk -F ';' '($2==1){print}' data.dat > data1.dat Just change the value in the $2== condition. Or, if you want to do this automatically, just use: awk -F ';' '{print > "data"$2".dat"}' data.dat which...

How to pivot array into another array in Ruby

arrays,ruby,csv

Here is a way using an intermediate hash-of-hash The h ends up looking like this {"Alaska"=>{"Rain"=>"3", "Snow"=>"4"}, "Alabama"=>{"Snow"=>"2", "Hail"=>"1"}} myArray = [["Alaska","Rain","3"],["Alaska","Snow","4"],["Alabama","Snow","2"],["Alabama","Hail","1"]] myFields = ["Snow","Rain","Hail"] h = Hash.new{|h, k| h[k] = {}} myArray.each{|i, j, k| h[i][j] = k } p [["State"] + myFields] + h.map{|k, v| [k] + v.values_at(*myFields)} output...

Panda's Write CSV - Append vs. Write

python,csv,pandas

Not sure there is a way in pandas but checking if the file exists would be a simple approach: import os # if file does not exist write header if not os.path.isfile('filename.csv'): df.to_csv('filename.csv',header ='column_names') else: # else it exists so append without writing the header df.to_csv('filename.csv',mode = 'a',header=False) ...

How to write a csv with quotes?

python,csv

Try ammending the csv.writer line to: writer=csv.writer(m,delimiter=',',quotechar=('\n"'),quoting=csv.QUOTE_ALL) See here for other quoting options....

create multidimensional associative array from CSV in PHP

php,csv,multidimensional-array,associative-array

This may help you Script - For csv to array from file [[email protected] tmp]$ cat test.php <?php function csv_to_array($filename='', $delimiter=',') { if(!file_exists($filename) || !is_readable($filename)) return FALSE; $header = NULL; $data = array(); if (($handle = fopen($filename, 'r')) !== FALSE) { while (($row = fgetcsv($handle, 0, $delimiter)) !== FALSE) { if(!$header)...

Python csv writer: write a string in front of an array, all in individual columns

python,csv

Think of what you want each row to look like, then build that row. In this case, you want the row to start with one element of names and have the elements of the corresponding element of matrix appended to it. Use zip() to combine the two sequences into one,...

Specific rows from CSV as dictionary and logic when keys are the same - Python

python,csv,dictionary

You can build the whole dictionary first, with the lists containing all the values for each key. Then once the dictionary is made, you can go through every key and take the largest and smallest values. yourdict = dict() with open(file) as f: filedata = f.read().splitlines() for line in filedata:...

Parse text from a .txt file using csv module

python,python-2.7,parsing,csv

How about using Regular Expression def get_info(string_to_search): res_dict = {} import re find_type = re.compile("Type:[\s]*[\w]*") res = find_type.search(string_to_search) res_dict["Type"] = res.group(0).split(":")[1].strip() find_Status = re.compile("Status:[\s]*[\w]*") res = find_Status.search(string_to_search) res_dict["Status"] = res.group(0).split(":")[1].strip() find_date = re.compile("Date:[\s]*[/0-9]*") res = find_date.search(string_to_search) res_dict["Date"] = res.group(0).split(":")[1].strip() res_dict["description"] =...

Print lines in a csv file if the values are consecutive

python,csv

The big problem, you have to change & to the word and. Read CoryKramers comment for more of an explanation as to why. A couple more things, it is always better to use with open when dealing with files. You want to skip the first line, since it is just...

How can I use a variable to get an Input$ in Shiny?

r,variables,csv,shiny

input is just a reactivevalues object so you can use [[: print(input[[a]]) ...

Replace improper commas in CSV file

regex,r,csv

If you need the comments, you still can replace the 6th comma with a semicolon and use your previous solution: gsub("((?:[^,]*,){5}[^,]*),", "\\1;", vec1, perl=TRUE) Regex explanation: ((?:[^,]*,){5}[^,]*) - a capturing group that we will reference to as Group 1 with \\1 in the replacement pattern, matching (?:[^,]*,){5} - 5 sequences...

writing csv output python

python,csv

Something like this should work for your question: import csv import calendar from collections import defaultdict months = [calendar.month_name[i] for i in range(0, 13)] totals = defaultdict(int) with open("data.csv", "r") as inf, open("data-out.csv", "w") as ouf: reader = csv.DictReader(inf) writer = csv.DictWriter(ouf, ['Name'] + months[5:9]) writer.writeheader() for row in reader:...

Addition of two dates on python 3

python,csv,datetime,python-3.x

Once you have both the values in two variables, new_date and new_time, you could simply combine them to get the datetime, like this, >>> new_date = dt.datetime.strptime(row[0], "%Y.%m.%d") >>> new_time = dt.datetime.strptime(row[1], "%H:%M").time() >>> >>> dt.datetime.combine(new_date, new_time) datetime.datetime(2005, 2, 28, 17, 38) Note:- Avoid using date and time as variable...

Resampling and merging data frame with python

python,csv,pandas,resampling,merging-data

You can concat the two DataFrames, interpolate, then reindex on the DataFrame you want. I assume we have a certain number of DataFrames, where the Date is a DateTimeIndex in all of them. I will use two in this example, since you used two in the question, but the code...

Convert strings of data to “Data” objects in R [duplicate]

r,date,csv

If you read on the R help page for as.Date by typing ?as.Date you will see there is a default format assumed if you do not specify. So to specify for your data you would do nmmaps$date <- as.Date(nmmaps$date, format="%m/%d/%Y") ...

Python Datetime into MySQL

python,mysql,csv,datetime

I took your code, and it seems like the main issues are in the append_to_database function. It looks like the return value of normalise_date_to_UTF returns a datetime object, which is being passed to append_to_database (as the timestampval argument), and that is being run through strptime, which is unnecessary since it's...

Import csv file and update existing table in Postgres

postgresql,csv,heroku-postgres,csv-import

SQL>begin; BEGIN Time: 0.366 ms SQL>truncate table t; TRUNCATE TABLE Time: 3.068 ms SQL>select * from t; t --- (0 rows) Time: 2.844 ms SQL>copy t from '/tmp/t'; COPY 2 Time: 1.693 ms SQL>select * from t; t ------------------------------- 2014-10-09 08:09:58.241592+00 2015-06-17 09:18:05.731139+00 (2 rows) Time: 1.823 ms SQL>end; COMMIT...

How can i add two tables in excel csv file?

c#,excel,csv,export

Well, CSV stands for comma separated values, if you open up your generated file through note pad you will have a real look what those data really looks like and you will have an idea on what your asking for. to insert empty row you just do "","","","" To insert...