I have a matrix of data 29523 rows x 503 cols of which 3 cols are indices (below is a subset for example).

```
IDX1| IDX2 | IDX3 | 1983 Q4 | X | Y | Z |1984 Q1 | X | Y | Z
---------------------------------------------------------------------------
A | A1 | Q | 10 | A | F | NaN | 110 | A | F | NaN
A | A2 | Q | 20 | B | C | 40 | 120 | B | C | 240
A | A3 | Q | 30 | A | F | NaN | 130 | A | F | NaN
A | A4 | Q | 40 | B | C | 80 | 140 | B | C | 280
A | A5 | Q | 50 | A | F | NaN | 150 | A | F | NaN
A | A6 | Q | 60 | B | F | 120 | 160 | B | F | 320
```

I read this into a `DataFrame`

with:

```
>>> df = pd.read_csv(C:\filename.csv, low_memory=False, mangle_dupe_cols=False)
```

and then use `pandas.melt()`

to pivot the data:

```
df1 = pd.melt(df, id_vars=['IDX1', 'IDX2', 'IDX3'], var_name='ValueType',
value_name = 'Value')
```

I have also tried `stack()`

but `melt()`

proved better here.

```
IDX1 | IDX2 | IDX3 | ValueType | Value
---------------------------------------------------------------
A | A1 | Q | 1983 Q4 | 10
A | A1 | Q | X | A
A | A1 | Q | Y | F
A | A1 | Q | Z | NaN
A | A1 | Q | 1984 Q1 | 110
A | A1 | Q | X | A
A | A1 | Q | Y | F
A | A1 | Q | Z | NaN
A | A2 | Q | 1983 Q4 | 20
A | A2 | Q | X | B
A | A2 | Q | Y | C
A | A2 | Q | Z | 40
```

The option `mangle_dupe_cols`

on the `read_csv`

if `True`

will place a `.int`

suffix against all `ValueType`

s that are duplicated. This is not ideal, but without it there is no way of linking the values for the variables to the correct period.

What I would prefer to do is instead of having the `Period`

`(1984 Q1)`

as a `ValueType`

, give the `Period`

s corresponding `Value`

a variable `'W'`

and have each period form part of the `IDX`

as below:

```
IDX1 | IDX2 | IDX3 | IDX4 | ValueType | Value
---------------------------------------------------------------
A | A1 | Q | 1983 Q4| W | 10
A | A1 | Q | 1983 Q4| X | A
A | A1 | Q | 1983 Q4| Y | F
A | A1 | Q | 1983 Q4| Z | NaN
A | A1 | Q | 1984 Q1| W | 110
A | A1 | Q | 1984 Q1| X | A
A | A1 | Q | 1984 Q1| Y | F
A | A1 | Q | 1984 Q1| Z | NaN
A | A2 | Q | 1983 Q4| W | 20
A | A2 | Q | 1983 Q4| X | B
A | A2 | Q | 1983 Q4| Y | C
A | A2 | Q | 1983 Q4| Z | 40
```

Is the above possible with pandas or numpy?

My final `DataFrame`

is going to be 14,761,500 rows x 6 cols.