12/9 Has anyone had experience with using a file under NFS as basically
a region of memory shared between separate machines? I.e., the
machines lock the file and read/write it to communicate. Speed is
not a huge concern, but can this be done reliably? Are there
caching issues? Will fsync work properly over NFS? This doesn't
seem ideal, but thanks for any advice.
\- you may be interested in RDMA, which is something i am intrested in.
however, here this issue is speed and offloading from the processor.
if you are interested, you can send me a note, however i am a little
busy these days so i may not have a lot of time to discuss.
jeff mogul's paper on tcp offloading has some decent references
for background. --psb
\_ for extra bonus points, use mmap() to get a truly shared memory
region over the network.
\_ I've done something like this before. If you are not on the
same switch (best to make a vlan w/ just two machines) you
may have problems. If nfsd is setup to do caching you might
run into issues. You will most likely have problems between
Linux (b/c linux nfs sux) and any other os. Try and stick to
Solaris and FreeBSD (or MacOS X) if you really want this to
work. BTW, why are you even thinking about this?
\- oh it finally ocurrs to me what you are trying to do ...
you are trying to use a file as a way of talking between
processes ... in this case the processes are on different
machines and the file happens to be nfs mounted. oh jesus.
\_ Sucks as of when? I know Linux NFS used to be pretty bad, but
I've heard it has improved a lot in recent years. Does anyone
know of other approaches used successfully in distributed
processing sort of applications (yes, I will STFW, but
corroboration is nice too)? It needs to be portable
across modern Unixes, and be able to handle moderate lock
contention (say, up to a couple hundred or so processes
vying for the lock -- essentially, the shared data is just
a job number though, so it's pretty small). Thanks. -op
\_ Have you looked at MPI or PVM? -tom
\- MPI (MPICH, LAM/MPI) is the standard.
Get with the program. You could use
JavaSpaces. I think you get a free
"I am an idiot" tshirt with every
download. --psb
\_ Linux NFS has sucked and continues to suck, even with
2.6 kernel (esp. if you stick to a kernel that comes
with RH or SuSE, b/c you want support &c.). I have
to deal with Linux NFS on almost a daily basis b/c
we have a unified Jumpstart/KickStart/AutoYAST server
as part of our product and whenever anyone tries to
boot systems (more than 1) using NFS, Linux starts
having problems with NFS (both userland and kernel
nfs have problems).
NFS v3 has problems talking to most other NFS v3
implementations. You will start seeing weird file
corruption, hanging mounts, blocking reads and stuff
after a while. You could try NFS v2, but Linux has
problems doing TCP and NFS v2, and for this sort of
thing you really want TCP.
If you are trying to do some sort of parallel clustering
thing, take a look at something like GridWare.
[ Sorry, if it sounds like I'm babbling, but I just
got done with a 3+ hr final and I'm a bit tired ]
\- nfsv4 opensrc implementation is being mainly done
on linux by umich/citi which has some pretty clever
folks so the v4 implementation may be bounded non-ass.
the connectathon results look decent. by "bounded"
i mean "as good as something can be on AssOS". --psb
\_ I agree that NFSv4 will probably be pretty
good, but AFAIK only Linux and Solaris 10
have a working version right now and he
said modern Unices by which I assumed he
ment stuff like HPUX 10, 11, AIX [4?],
FreeBSD 4.x-5.3, Linux 2.[2-6], Solaris
2.6-10 and MacOS X (NetBSD and OpenBSD
don't make my list b/c hardly anyone
uses them for general purpose stuff).
\- i think there are some connectathon
summaries from about a month ago on
one of the citi WEEB sites ... nothing
dramatic there but if you are really
interested in the details about the
state of affairs.
\_ Why on the same switch/VLAN? Any idea why that makes
a difference?
\_ [ It was been a little while, so take this with
a grain of salt ]
The switch has to forward all broadcast traffic
to every port active port on a VLAN. By default
every system plugged into the switch is on the
same default VLAN (1 for Cisco, iirc). If you
have lots of other machines on the same switch
a the two systems that are using NFS, then the
extra broadcast traffic can affect your network
performance. By making a separate VLAN you are
removing this potential problem.
\_ What does NFS have to do with broadcast traffic? |
http://csua.com/feed/