我正在尝试从文件中读取特定行并在结束每个 block 的进程后继续读取。 比方说,我在一个文件中有 19000 行。每次,我都会提取前 19 行,对这些行进行一些计算并将输出写入另一个文件。然后我将再次提取接下来的 19 行并进行相同的处理。因此,我尝试通过以下方式提取行:
n=19
x = defaultdict(list)
i=0
fp = open("file")
for next_n_lines in izip_longest(*[fp] *n):
lines = next_n_lines
for i, line in enumerate(lines):
do calculation
write results
这里的代码适用于第一个 block 。你们中的任何人都可以帮助我,我该如何继续下一个 n block ?非常感谢!
请您参考如下方法:
您的代码已经以 19 行为一组提取行,所以我不确定您的问题是什么。
我可以稍微清理一下您的解决方案,但它的作用与您的代码相同:
from itertools import izip_longest
# grouping recipe from itertools documentation
def grouper(n, iterable, fillvalue=None):
"Collect data into fixed-length chunks or blocks"
# grouper(3, 'ABCDEFG', 'x') --> ABC DEF Gxx
args = [iter(iterable)] * n
return izip_longest(fillvalue=fillvalue, *args)
def process_chunk(chunk):
"Return sequence of result lines. Chunk must be iterable."
for i, line in enumerate(chunk):
yield 'file-line {1:03d}; chunk-line {0:02d}\n'.format(i, int(line))
yield '----------------------------\n'
下面是一些测试代码,演示了每一行都被访问了:
from StringIO import StringIO
class CtxStringIO(StringIO):
def __enter__(self):
return self
def __exit__(self, *args):
return False
infile = CtxStringIO(''.join('{}\n'.format(i) for i in xrange(19*10)))
outfile = CtxStringIO()
# this should be the main loop of your program.
# just replace infile and outfile with real file objects
with infile as ifp, outfile as ofp:
for chunk in grouper(19, ifp, '\n'):
ofp.writelines(process_chunk(chunk))
# see what was written to the file
print ofp.getvalue()
此测试用例应打印如下行:
file-line 000; chunk-line 00
file-line 001; chunk-line 01
file-line 002; chunk-line 02
file-line 003; chunk-line 03
file-line 004; chunk-line 04
...
file-line 016; chunk-line 16
file-line 017; chunk-line 17
file-line 018; chunk-line 18
----------------------------
file-line 019; chunk-line 00
file-line 020; chunk-line 01
file-line 021; chunk-line 02
...
file-line 186; chunk-line 15
file-line 187; chunk-line 16
file-line 188; chunk-line 17
file-line 189; chunk-line 18
----------------------------
