From: Ansgar Burchardt <ansgar@debian.org>
Date: Fri, 15 Apr 2016 07:49:54 +0000 (+0200)
Subject: Read files in chunks
X-Git-Url: https://git.decadent.org.uk/gitweb/?a=commitdiff_plain;h=653c6617323bc6601c071185265a5f8b663e5e3d;p=dak.git

Read files in chunks

Calling `f.read()` would read the entire file at once which is
wasteful. It's also not intended given the read loop.

Also replace a use of `readline()` with `read(65536)` as well where we
want to read the entire file and not only single lines.
---

diff --git a/dak/generate_index_diffs.py b/dak/generate_index_diffs.py
index 6637c763..485d3770 100755
--- a/dak/generate_index_diffs.py
+++ b/dak/generate_index_diffs.py
@@ -107,7 +107,7 @@ def smartopen(file):
 def pipe_file(f, t):
     f.seek(0)
     while 1:
-        l = f.read()
+        l = f.read(65536)
         if not l: break
         t.write(l)
     t.close()
@@ -243,7 +243,7 @@ class Updates:
 def create_temp_file(r):
     f = tempfile.TemporaryFile()
     while 1:
-        x = r.readline()
+        x = r.read(65536)
         if not x: break
         f.write(x)
     r.close()