Skip to main content
 首页 » 编程设计

python之Pandas:具有多索引的 fillna() 方法之NaN 填充了错误的列

2025年05月04日10zlslch

这是一个重现我的问题的工作示例。首先生成一些随机数据以及我们将用于填充 nans 的数据:

#Generate some random data and data that will be used to fill the nans 
data = np.random.random((100,6)) 
fill_data = np.vstack((np.ones(200), np.ones(200)*2, np.ones(200)*3,np.ones(200), np.ones(200)*2, np.ones(200)*3)).T 
 
#Generate indices of nans that we will put in 
nan_rows = np.random.randint(0,100,50) 
nan_cols = np.random.randint(0,6,50) 
nan_idx = np.vstack((nan_rows,nan_cols)).T 
 
#Put in nan values 
for r,c in nan_idx: 
    data[r,c] = np.nan 
 
#Generate multiindex and datetimeindex for both the data and fill_data 
multi = pd.MultiIndex.from_product([['A','B'],['one','two','three']]) 
idx1 = pd.DatetimeIndex(start='1990-01-01', periods=100, freq='d') 
idx2 = pd.DatetimeIndex(start='1989-12-01', periods=200, freq='d') 
 
#Construct dataframes 
df1 = pd.DataFrame(data, idx1, multi) 
df2 = pd.DataFrame(fill_data, idx2, multi) 
 
#fill nans from df1 with df2 
df1 = df1.fillna(df2, axis=1) 

这是生成的帧的样子:

In [167]: 
 
df1.head() 
 
Out[167]: 
            A                                   B 
            one         two         three       one         two         three 
1990-01-01  1.000000    0.341803    0.694128    0.382164    0.326956    0.506616 
1990-01-02  0.439024    0.552746    0.538489    0.003906    0.968498    0.816289 
1990-01-03  0.200252    0.838014    0.805633    0.008980    0.269189    0.016243 
1990-01-04  0.735120    0.384871    0.579268    0.561657    0.630314    0.361932 
1990-01-05  0.938185    0.335212    0.678310    2.000000    0.819046    0.482535 
 
 
 
In [168]: 
 
df2.head() 
 
Out[168]: 
    A   B 
    one     two     three   one     two     three 
1989-12-01  1   2   3   1   2   3 
1989-12-02  1   2   3   1   2   3 
1989-12-03  1   2   3   1   2   3 
1989-12-04  1   2   3   1   2   3 
1989-12-05  1   2   3   1   2   3 

所以这里的关键是数据帧的长度不同但具有共同的标签,因为多索引列是相同的并且 df1 中的时间戳标签在 df2 中。

结果如下:

In [165]: 
 
df1 
 
Out[165]: 
            A                                   B 
            one         two         three       one         two         three 
1990-01-01  1.000000    0.341803    0.694128    0.382164    0.326956    0.506616 
1990-01-02  0.439024    0.552746    0.538489    0.003906    0.968498    0.816289 
1990-01-03  0.200252    0.838014    0.805633    0.008980    0.269189    0.016243 
1990-01-04  0.735120    0.384871    0.579268    0.561657    0.630314    0.361932 
1990-01-05  0.938185    0.335212    0.678310    2.000000    0.819046    0.482535 
1990-01-06  0.609736    0.164815    0.295003    0.784388    3.000000    3.000000 
1990-01-07  1.000000    0.394105    0.430608    0.782029    0.327485    0.855130 
1990-01-08  0.573780    0.525845    0.147302    0.091022    3.000000    3.000000 
1990-01-09  0.591646    0.651251    0.649255    0.205926    3.000000    0.606428 
1990-01-10  0.988085    0.524769    0.481834    0.486241    0.629223    0.575843 
1990-01-11  1.000000    0.586813    0.592252    0.309429    0.877121    0.547193 
1990-01-12  0.853000    0.097981    0.970053    0.519838    0.828266    0.618965 
1990-01-13  0.579778    0.805140    0.050559    0.432795    0.036241    0.081218 
1990-01-14  0.055462    1.000000    0.159151    0.538137    3.000000    0.296754 
1990-01-15  0.848238    0.697454    0.519403    0.232734    0.612487    0.891230 
1990-01-16  0.808238    0.182904    0.480846    0.052806    0.900373    0.860274 
1990-01-17  0.890997    0.346767    0.265168    0.486746    0.983999    0.104035 
1990-01-18  0.673155    0.248853    0.245246    2.000000    0.965884    0.295021 
1990-01-19  0.074864    0.714846    2.000000    0.046031    0.105930    0.641538 
1990-01-20  1.000000    0.486893    0.464024    0.499484    0.794107    0.868002 

如果仔细观察,您会发现列 ('A','one') 和 ('A','two') 中的值等于 1,('A',' 中的值等于 2 three') 和 ('B','one') 以及在 ('B','two') 和 ('B','three') 中等于 3 的值。

预期输出将是“一”列中的值 1,“二”列中的值 2,等等。

我是不是做错了什么?对我来说,这似乎是某种错误。

请您参考如下方法:

这issue has been fixed在最新版本的 Pandas 中。

使用 0.15.0 版你将能够做到这一点:

import pandas as pd 
import numpy as np 
from numpy import nan     
 
df = pd.DataFrame({'a': [nan, 1, 2, nan, nan], 
                   'b': [1, 2, 3, nan, nan], 
                   'c': [nan, 1, 2, 3, 4]}, 
                  index = list('VWXYZ')) 
#     a   b   c 
# V NaN   1 NaN 
# W   1   2   1 
# X   2   3   2 
# Y NaN NaN   3 
# Z NaN NaN   4 
 
# df2 may have different index and columns 
df2 = pd.DataFrame({'a': [10, 20, 30, 40, 50], 
                    'b': [50, 60, 70, 80, 90], 
                    'c': list('ABCDE')}, 
                   index = list('VWXYZ')) 
#     a   b  c 
# V  10  50  A 
# W  20  60  B 
# X  30  70  C 
# Y  40  80  D 
# Z  50  90  E 

现在,将 DataFrame 传递给 fillna

result = df.fillna(df2) 

产量

print(result) 
#     a   b  c 
# V  10   1  A 
# W   1   2  1 
# X   2   3  2 
# Y  40  80  3 
# Z  50  90  4