File 0001-sftp-bump-pending-request-count-from-1-to-... of Package kio-extras5

Overview Repositories Revisions Requests Users Attributes Meta

File 0001-sftp-bump-pending-request-count-from-1-to-128.patch of Package kio-extras5 (Revision 0baa1dcaa3531fc7f0bc11ad94bb5736)

Currently displaying revision 0baa1dcaa3531fc7f0bc11ad94bb5736 , Show latest

From 40d962d80b6f2e9d04428c5d1944834de1208fac Mon Sep 17 00:00:00 2001
From: Harald Sitter <sitter@kde.org>
Date: Wed, 12 Sep 2018 16:58:04 +0200
Subject: [PATCH] [sftp] bump pending request count from 1 to 128

Summary:
with the previous value we basically did sync reading which means that
network and cyrptographic overhead head a huge impact on throughput.
meanwhile the perfect way to use asyncness is to schedule a whole bunch of
requests before starting to read.
previously this was documented as auto-adjusting, which it never was,
there's also little to be gained from adjusting this value on the fly.
more requests in most scenarios will simply mean a larger RAM footprint as
more data potentially sits in libssh waiting to be read. with 128 requests
that'd be ~8mb (assuming the file transferred is that large)

this improves read performance with libssh 0.8 by up to 20 times for large
files. read performance with libssh 0.6 is 2 to 3 times better.
the faster the connection the higher the gain of course.

128 gives somewhat competitive performance results compared to openssh's
ssh implementations while not having too large a footprint.

in raw numbers: a local link read was averaging around 10mb/s on both
libssh versions. with 128 requests this goes up to 200. competitive is
between 200 and 300 it seems (obviously all specific to my system)

CHANGELOG: sftp file reading is now up to 20 times faster

Reviewers: broulik

Reviewed By: broulik

Differential Revision: https://phabricator.kde.org/D15452
---
 sftp/kio_sftp.h | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/sftp/kio_sftp.h b/sftp/kio_sftp.h
index cc6b9e09..e5639970 100644
--- a/sftp/kio_sftp.h
+++ b/sftp/kio_sftp.h
@@ -141,10 +141,11 @@ private: // Private variables
      * @param file the sftp_file object which should be transferred.
      * @param sb the attributes of that sftp_file object.
      * @param maxPendingRequests the maximum number of parallel requests to start with.
-     *                            The number will be adjusted automatically depending
-     *                            on the connection speed.
+     *                           The more are pending the higher the potential memory
+     *                           foot print, however if the connection allows it
+     *                           we'll get better throughput.
      */
-    GetRequest(sftp_file file, sftp_attributes sb, ushort maxPendingRequests = 1);
+    GetRequest(sftp_file file, sftp_attributes sb, ushort maxPendingRequests = 128);
     /**
      * Removes all pending requests and closes the SFTP channel and attributes
      * in order to avoid memory leaks.
-- 
2.18.0

Places

File 0001-sftp-bump-pending-request-count-from-1-to-128.patch of Package kio-extras5 (Revision 0baa1dcaa3531fc7f0bc11ad94bb5736)

Places