Say I have two pandas Series in python:
import pandas as pd h = pd.Series(['g',4,2,1,1]) g = pd.Series([1,6,5,4,"abc"])
I can create a DataFrame with just h and then append g to it:
df = pd.DataFrame([h]) df1 = df.append(g, ignore_index=True)
>>> df1 0 1 2 3 4 0 g 4 2 1 1 1 1 6 5 4 abc
But now suppose that I have an empty DataFrame and I try to append h to it:
df2 = pd.DataFrame() df3 = df2.append(h, ignore_index=True)
This does not work. I think the problem is in the second-to-last line of code. I need to somehow define the blank DataFrame to have the proper number of columns.
By the way, the reason I am trying to do this is that I am scraping text from the internet using requests+BeautifulSoup and I am processing it and trying to write it to a DataFrame one row at a time.
So if you don’t pass an empty list to the DataFrame constructor then it works:
In : df = pd.DataFrame() h = pd.Series(['g',4,2,1,1]) df = df.append(h,ignore_index=True) df Out: 0 1 2 3 4 0 g 4 2 1 1 [1 rows x 5 columns]
The difference between the two constructor approaches appears to be that the index
dtypes are set differently, with an empty list it is an
Int64 with nothing it is an
In : df = pd.DataFrame() print(df.index.dtype) df = pd.DataFrame() print(df.index.dtype) object int64
Unclear to me why the above should affect the behaviour (I’m guessing here).
After revisiting this I can confirm that this looks to me to be a bug in pandas version
0.12.0 as your original code works fine:
In : import pandas as pd df = pd.DataFrame() h = pd.Series(['g',4,2,1,1]) df.append(h,ignore_index=True) Out: 0 1 2 3 4 0 g 4 2 1 1 [1 rows x 5 columns]
I am running pandas
0.13.1 and numpy
1.8.1 64-bit using python
184.108.40.206 but I think the problem is pandas but I would upgrade both pandas and numpy to be safe, I don’t think this is a 32 versus 64-bit python issue.