File 0001-rename-local-preload-function-to-not-overw... of Package borgbackup

Overview Repositories Revisions Requests Users Attributes Meta

File 0001-rename-local-preload-function-to-not-overwrite-keyword-argument.patch of Package borgbackup (Revision 2692c9bb8e2ee50cdcb0455232dd087e)

Currently displaying revision 2692c9bb8e2ee50cdcb0455232dd087e , Show latest

From dad3aa9dae77cb38b1559c5ac8a082d3f8658beb Mon Sep 17 00:00:00 2001
From: Elmar Hoffmann <elho@elho.net>
Date: Mon, 1 Jun 2020 13:22:25 +0200
Subject: [PATCH] rename local preload() function to not overwrite keyword
 argument of same name

The locally defined preload() function overwrites the preload boolean keyword
argument, always evaluating to true, so preloading is done, even when not
requested by the caller, causing a memory leak.
Also move its definition outside of the loop.

This issue was found by Antonio Larrosa in borg issue #5202.
---
 src/borg/archive.py | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/src/borg/archive.py b/src/borg/archive.py
index ee43d7012..7fdcec3ec 100644
--- a/src/borg/archive.py
+++ b/src/borg/archive.py
@@ -262,6 +262,9 @@ def unpack_many(self, ids, filter=None, partial_extract=False, preload=False, ha
         Warning: if *preload* is True then all data chunks of every yielded item have to be retrieved,
         otherwise preloaded chunks will accumulate in RemoteRepository and create a memory leak.
         """
+        def _preload(chunks):
+            self.repository.preload([c.id for c in chunks])
+
         masters_preloaded = set()
         unpacker = msgpack.Unpacker(use_list=False)
         for data in self.fetch_many(ids):
@@ -271,9 +274,6 @@ def unpack_many(self, ids, filter=None, partial_extract=False, preload=False, ha
                 if 'chunks' in item:
                     item.chunks = [ChunkListEntry(*e) for e in item.chunks]
 
-            def preload(chunks):
-                self.repository.preload([c.id for c in chunks])
-
             if filter:
                 items = [item for item in items if filter(item)]
 
@@ -286,7 +286,7 @@ def preload(chunks):
                     # due to a side effect of the filter() call, we now have hardlink_masters dict populated.
                     for item in items:
                         if 'chunks' in item:  # regular file, maybe a hardlink master
-                            preload(item.chunks)
+                            _preload(item.chunks)
                             # if this is a hardlink master, remember that we already preloaded it:
                             if 'source' not in item and hardlinkable(item.mode) and item.get('hardlink_master', True):
                                 masters_preloaded.add(item.path)
@@ -296,13 +296,13 @@ def preload(chunks):
                                 # we only need to preload *once* (for the 1st selected slave)
                                 chunks, _ = hardlink_masters[source]
                                 if chunks is not None:
-                                    preload(chunks)
+                                    _preload(chunks)
                                 masters_preloaded.add(source)
                 else:
                     # easy: we do not have a filter, thus all items are selected, thus we need to preload all chunks.
                     for item in items:
                         if 'chunks' in item:
-                            preload(item.chunks)
+                            _preload(item.chunks)
 
             for item in items:
                 yield item

Places

File 0001-rename-local-preload-function-to-not-overwrite-keyword-argument.patch of Package borgbackup (Revision 2692c9bb8e2ee50cdcb0455232dd087e)

Places