抱歉,标题含糊不清,但因为我真的不知道问题是什么...问题是我想加载一个 CSV 文件,然后将它分成两个数组并在每个数组上执行一个函数那些数组。它适用于第一个阵列,但第二个阵列出现问题,即使一切都相同。我真的卡住了。代码如下:
from wordutility import wordutility
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn import cross_validation
from sklearn.naive_bayes import MultinomialNB
from sklearn.svm import LinearSVC
import pandas as pd
import numpy as np
data = pd.read_csv('sts_gold_tweet.csv', header=None, delimiter=';',
quotechar='"')
# test = pd.read_csv('output.csv', header=None,
# delimiter=';', quotechar='"')
split_ratio = 0.9
train = data[:round(len(data)*split_ratio)]
test = data[round(len(data)*split_ratio):]
y = data[1]
print("Cleaning and parsing tweets data...\n")
traindata = []
for i in range(0, len(train[0])):
traindata.append(" ".join(wordutility.tweet_to_wordlist
(train[0][i], False)))
testdata = []
for i in range(0, len(test[0])):
testdata.append(" ".join(wordutility.tweet_to_wordlist(test[0][i], False)))
程序运行到最后一行。错误是:
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "/usr/lib/python3.4/site-packages/pandas/core/series.py", line 509, in __getitem__
result = self.index.get_value(self, key)
File "/usr/lib/python3.4/site-packages/pandas/core/index.py", line 1417, in get_value
return self._engine.get_value(s, k)
File "pandas/index.pyx", line 100, in pandas.index.IndexEngine.get_value (pandas/index.c:3097)
File "pandas/index.pyx", line 108, in pandas.index.IndexEngine.get_value (pandas/index.c:2826)
File "pandas/index.pyx", line 154, in pandas.index.IndexEngine.get_loc (pandas/index.c:3692)
File "pandas/hashtable.pyx", line 381, in pandas.hashtable.Int64HashTable.get_item (pandas/hashtable.c:7201)
File "pandas/hashtable.pyx", line 387, in pandas.hashtable.Int64HashTable.get_item (pandas/hashtable.c:7139)
KeyError: 0
(它在错误代码中说第 2 行,因为我在 python shell 中尝试代码。所以第 2 行指的是上面代码的最后一行。)
希望有人能帮助我 :)。 谢谢
编辑
好吧,看来 split 并没有像我想象的那样奏效。我确实得到了我想要的两个数组,但不知何故,这些行仍然像一个文件一样。所以数组 train 是从 0 到 1830,数组 test 是从 1831 到 2034...所以范围是错误的...我将如何“正确”拆分 csv 文件?
2 编辑
>>> print(train[0:5])
0 1
0 the angel is going to miss the athlete this we... negative
1 It looks as though Shaq is getting traded to C... negative
2 @clarianne APRIL 9TH ISN'T COMING SOON ENOUGH negative
3 drinking a McDonalds coffee and not understand... negative
4 So dissapointed Taylor Swift doesnt have a Twi... negativ
>>> print(test[0:5])
0 1
1831 Why is my PSP always dead when I want to use it? negative
1832 @hillaryrachel oh i know how you feel. i took ... negative
1833 @daveknox awesome- corporate housing took awa... negative
1834 @lakersnation Is this a joke? I can't find them negative
1835 XBox Live still down negative
如您所见,数组“test”从行号 1831 开始。我原以为它会从 0 开始……我现在通过编辑 for 循环中的范围解决了我的问题
for i in range(len(train[0], len(data)):
所以我原来的问题已经解决了,我只是好奇并且渴望学习编写更好的代码。这是一件好事还是我应该以不同的方式拆分 csv 文件?
请您参考如下方法:
当你执行 test[0] 时,你没有得到 test 的第一个索引,它更像是你得到 test< 的列 与“名称”0。当您将 pandas DataFrame 一分为二时,原始列名将被保留。这意味着对于 test DataFrame,它没有列 0,因为该列在第一个 DataFrame 中。
让我举个例子。假设您有以下 DataFrame:
0 1 2 3 4 5 6 7 8 9
Ind1 0 1 2 3 4 5 6 7 8 9
Ind2 10 11 12 13 14 15 16 17 18 19
当你拆分它时,你最终会得到这些 DataFrame:
0 1 2 3 4
Ind1 0 1 2 3 4
Ind2 10 11 12 13 14
和:
5 6 7 8 9
Ind1 5 6 7 8 9
Ind2 15 16 17 18 19
请注意,第二个 DataFrame 的列以 5 开头,而不是 0,因为它们是拆分前的列名。因此,当您尝试获取列 0 时,它不存在。那是你错误的根源。
最简单的解决方案就是使用索引,而不是列名。因此,不要使用 test[0],而是使用 test.iloc[0]。这将根据位置索引给出值。
