On Sat, 2 Jan 2010, Peter Stuge wrote:
>> When you want to handle multiple streams, you use non-blocking
>> APIs.
>
> Or I could use select() together with blocking I/O.
You say this, but...
your definition of blocking operations that always do "short" reads and writes
and never gets stuck on anything is virtually the same thing as non-blocking
sockets so your blocking approach is hardly distinguishable from our current
non-blocking way. Can you describe in clear terms how your "blocking i/o"
would differ? If the calls always return write() and read() early and never
blocks on any call that use the socket, then what's the difference and why
does that make anything easier to an application?
I didn't invent non-blocking I/O and non-blocking has a purpose and has been
used for ages. It is guaranteed to not block, while the blocking one can block
at times. Multiplexing many streams CANNOT be done flawlessly and optimally in
a single thread with blocking I/O. It can be made to work, yes. But that's not
the same thing.
>> That's not something libssh2 has invented, it's a well known and well used
>> paradigm since ages.
>
> Yes, and that is true also for blocking I/O with select(). For me personally
> that is even more well known than non-blocking.
Yes, you can multiplex with blocking I/O and it's been done for ages and it
mostly works too without problems. I wrote my first blocking I/O codes using
select() in the early 90s. But during all this time there have been blocking
and non-blocking operations. When we need to pick a blocking or a non-blocking
call, it is kind of explicit that a call that blocks will ruin multiplexing to
some degree. Quite possibly not significantly or even noticably in most cases,
but blocking is blocking and blocking when an alternative would be
non-blocking hampers multi-plexing.
Again, my understanding is that you say blocking I/O acts like the
non-blocking I/O case so thus we won't gain anything from switching to
blocking anyway...
>> Why is this a problem at all?
>
> The problem is that libssh2 design makes it impossible to use blocking I/O
I don't understand, but my question wasn't why you can't use blocking but why
non-blocking is a problem.
libssh2 provides a blocking and a non-blocking I/O API, why does that make it
impossible to use blocking I/O?
> - AFAIU only because you have bad experience from what sounds to me like a
> very broken select() implementation in an as-yet unnamed operating system.
> (Or several ones?)
Nope. Non-blocking I/O goes far beyond select() or any other single syscall.
Non-blocking means that no operation on the file handle ever blocks. Blocking
I/O implies that it can sometimes block. What else is the difference between
blocking and non-blocking mode?
And again: your so called blocking I/O is never blocking anyway according to
you so then by definition it will be non-blocking and that is exactly what we
have today.
> If you doubt that blocking is impossible
I've never said that blocking is impossible. It is certainly possible, just
inferior for the purposes where I use libssh2.
> I originally wrote the example using blocking I/O in order to keep it as
> simple as possible, but unfortunately it just can't work with the current
> API.
But we have a blocking API if you want it to be blocking. Why isn't that
working? Isn't _that_ the problem we should fix?
> I do like the idea to move I/O out of libssh2 and have the library do
> processing only.
That could be a cool approach, but couldn't we basically just provide an API
that allows an app to provide its own recv() and send() functions?
Having the lib work on a buffer to generate a full output buffer to get
returned (without any kind of callback) will be harder and possibly also cause
even more memory copies.
-- / daniel.haxx.se _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-develReceived on 2010-01-02