11/13 If I have a application on machine foo sending data really fast to
an application on machine bar via TCP, and the bar applications job
is write that to disk as fast as possible, what happens if the
network stream is faster than the disk writes?
\_ Look up the differences between TCP/IP flow control and
congestion control. The answer is in front of you.
http://en.wikipedia.org/wiki/Flow_control
http://web.mit.edu/~6.033/www/papers/Networks-5-FlowCongestion.pdf
By the way I hope we're not doing your EE122 homework.
\_ bar starts blocking on i/o and the requests will queue up.
Eventually the disk queue will fill up and how bar handles this
depends on the o/s.
\_ Basically all the backlog ends up in the memory of bar, right?
\_ It depends on 100's of things. Is this a single threaded
"read, write, repeat" application? If so your tcp buffers
on bar will eventually fill up, yes. How exactly depends on
the os you are using. Once that happens it will stop acking
packets and that means your buffers on foo will start to
fill up. Once the buffers on foo fill up (once again how/when
is going to depend on the os) your app on foo will probably
start blocking or returning an error because it would block.
Guess what, all that sort of behavior is also os dependant.
\_ Does send() start returning errors? Thx. -- !OP
\_ Depends on how you set up your socket. It will probably
block instead.
\_ Your socket will either block until it can send the data
or the send() will return errors, usually of either
EWOULDBLOCK or ENOMEM. -ERic
\_ I know syslog is udp, but how does syslogd handle a scenario
like this?
\_ dropped logs. Such is the nature of using udp... -ERic
\_ And it's intentional. When everything is going to hell
you don't want your syslogs adding to that hell by
forcing resends of tons of packets. Syslog needs to
fail without taking down the rest of the system.
\_ You really need to read Stevens. This stuff will be 100x clearer
then.
\_ I agree, though a word of warning about Stevens. It is (well,
they are) an excellent foundation text(s), but it will steer you
wrong if you're trying to write servers that can serve thousands
of javascript blocks embedding images and videos to social
networks like the market leader http://Slide.com, you can't use select.
The most common alternative I've seen is to use non-blocking
sockets, and poll them manually. And, yes, I know that CS 162
teaches you that polling is bad. The class lies. -dans
wrong if you're trying to write servers that can scale to
handle thousands of connections, you can't use select. The most
common alternative I've seen is to use non-blocking sockets, and
poll them manually. And, yes, I know that CS 162 teaches you
that polling is bad. The class lies. -dans
\_ Well, in the common case polling is bad. Occasionally it's
the right thing to do. No undergrad class is going to be
able to cover such a broad subject completely. -jrleek
\_ I agree it's a broad subject, but I disagree with teaching
ideas that are theoretically sound, but break down in
practice. Indeed, poll vs. select is subtle and probably
not something that needs be convered in CS 162, but I am,
nonetheless frustrated at the range of crippled products I
keep seeing because so many people just copy paste the
select loop from Stevens. I find it more appalling that
we encourage undergrads to use threads since a) most
people can't write working threaded code and b) the
performance hit for more than n threads is appalling
(typically n is 8, but it depends on your hardware). -dans
\- (the collective you) may want to look at the papers by
Gaurav Banga and Jeffrey Mogul and various coauthors
for a good low level discussion about select/poll.
influential papers. --psb
if you are not familar with kqueues, the kqueue paper
is also good. influential papers. --psb
\_ Thanks, I'll check those out. A lot of my thinking
on network performance is shaped by periodic
conversations with a handful of senior Cisco
engineers who keep leaving Cisco to found companies
and keep ending up back there because Cisco keeps
buying their companies. -dans
\_ Kqueues are awesome, and anyone who makes
makes recomendations about how to handle 1000s
of concurrent sockets but doesn't know about
kqueues (or other like implmentations) proves
just how usless his advice his.
\_ Do you have any Senior Cisco Engineers to
back you up?
\_ It's consistent with past performance but still
kind of amazing you could turn a journal reference
into something about you and your cool world of
high tech finance.
\_ Please take your meds. Your delusions are
getting the better of you. -dans
\_ oh lay off, dans' obnoxiousness at least makes the motd slightly
interesting. we need all the activity we can get, new undergrads
dont even know what vi is these days. |
http://csua.com/feed/