Subject: Re: Pipelining and recent sftp upload improvements

Re: Pipelining and recent sftp upload improvements

From: Will Cosgrove <>
Date: Fri, 3 Dec 2010 10:24:22 -0800

>> The advantages of this is that you can call write_async_begin say, 10 times in a row, then starting draining the acks using write_async, this minimizes network latency (aka sitting on a select() call) and does a better job maxing out available bandwidth.
> Advantage compared to the original naive implemention yes, but not to the current SFTP upload code, right?

I have not benchmarked the current code, but I would assume it's similar.

>> The disadvantage is it's more leg-work for the implementor because you have to track the offsets and drain replies manually.
> I actually only see disadvantages with this quirky API as I believe we can reach the same model of sending without forcing this onto our users.

The main reason I wrote it like this was to be able to merge with new releases without modifying the current API/code.

>> Thinking out loud, it might be worth adding a convenance API that takes a file path and does all this 'behind-the-scences' like the openssh do_upload/do_download methods do.
> Why? What is so bad with my current approach? It certainly maintains a very simple and familiar API to users (not to mention unmodified) and yet it should be possible to reach at least roughly the same throughput.

Your approach is good, I would have used it if I had known about it before I wrote my own API. :) But I still think a new API that does *all* the legwork of uploading/downloading a file would be useful. It seems like the logical next step if end users don't have to do anything but pass in a file path.


Received on 2010-12-03