Log in

No account? Create an account
Constantine A. Murenin [entries|archive|friends|userinfo]
Constantine A. Murenin

[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

netbsd 5 [Чт, 2009-11-05T17:00]
Constantine A. Murenin
[Tags|, ]
[Местонахождение |N2L 3G1]

turns out, netbsd is much worse than openbsd as far as network performance is concerned.

first, it appears that no window scaling is implemented or enabled by default on 5.0.

then, it appears that you can't even set the default tcp buffer size to 256K, or stuff simply won't work.

from there, it appears that after you do set the buffer size to something like 224K (net.inet.tcp.recvspace=229376), doing pkg_add kdebase4 will reach the mclpool limit, and your networking capabilities are kaput.

well, yeah, at least this fuck up is documented, but then note that the sysctl variable is not runtime modifiable (at least on i386), so you have to edit the /netbsd with gdb and reboot the system before you can actually have `pkg_add kdebase4` complete.

and, last but not least, i'm quite disappointed in the praised infrastructure of the netbsd project: there's a nice mirror at columbia.edu that I can get 100Mbps from, nyftp.netbsd.org (Columbia, NY, 19ms away), but it only carries the daily builds and the packages -- I cannot find any decent mirrors that carry git, cvs and the normal ftp files that I would get over 40Mbps from here at UWaterloo.

just fyi: as of now, at UW we have a 1Gbps link to the educational network (including all the US universities) and a 1Gbps link to the internet. both freebsd and openbsd do have the nearby educational mirrors that provide speeds of 100Mbps over here, ftp-archive.freebsd.org (at Buffalo, NY, 7ms away) and obsd.cec.mtu.edu (MI, 20ms away). note that to get 100Mbps with 224K buffer size you have to be at least as close as 19ms from the mirror.

100Mbps in action:
hype:tmp {562} date
Thu Nov  5 17:40:27 EST 2009
hype:tmp {563} ftp ftp://ftp-archive.FreeBSD.org/pub/FreeBSD/ISO-IMAGES-i386/8.0/
Connected to moss.cse.buffalo.edu.
220 ftp-archive.freebsd.org FTP server (Version wu-2.6.2(1) Fri Jun 12 09:41:08 EDT 2009) ready.
331 Guest login ok, send your complete e-mail address as password.
230 Guest login ok, access restrictions apply.
Remote system type is UNIX.
Using binary mode to transfer files.
200 Type set to I.
250 CWD command successful.
ftp> ls
150 Opening ASCII mode data connection for /bin/ls.
total 3664980
-rw-r--r--  1 89987  546    44777472 Oct 26 13:09 8.0-RC2-i386-bootonly.iso
-rw-r--r--  1 89987  546   655726592 Oct 26 13:10 8.0-RC2-i386-disc1.iso
-rw-r--r--  1 89987  546  1870997736 Oct 26 13:13 8.0-RC2-i386-dvd1.iso.gz
-rw-r--r--  1 89987  546   256112640 Oct 26 13:14 8.0-RC2-i386-livefs.iso
-rw-r--r--  1 89987  546   923340800 Oct 26 13:03 8.0-RC2-i386-memstick.img
-rw-r--r--  1 89987  546         326 Oct 26 16:14 CHECKSUM.MD5
-rw-r--r--  1 89987  546         501 Oct 26 16:14 CHECKSUM.SHA256
226 Transfer complete.
ftp> get 8.0-RC2-i386-bootonly.iso
local: 8.0-RC2-i386-bootonly.iso remote: 8.0-RC2-i386-bootonly.iso
150 Opening BINARY mode data connection for 8.0-RC2-i386-bootonly.iso (44777472 bytes).
100% |******************************************************************************************| 43728 KB    00:04    
226 Transfer complete.
44777472 bytes received in 4.05 seconds (10.54 MB/s)
ftp> quit
221-You have transferred 44777472 bytes in 1 files.
221-Total traffic for this session was 44778897 bytes in 2 transfers.
221-Thank you for using the FTP service on ftp-archive.freebsd.org.
221 Goodbye.
hype:tmp {564} traceroute ftp-archive.FreeBSD.org
traceroute to moss.cse.buffalo.edu (, 64 hops max, 40 byte packets
 1  dccore-nsfw02-csresearch1net.uwaterloo.ca (  0.452 ms  0.246 ms  0.300 ms
 2 (  0.596 ms  0.567 ms  0.536 ms
 3  v720-cn-rt-phy.uwaterloo.ca (  0.624 ms  0.556 ms  0.574 ms
 4  ext-rt-mc-cn-rt-phy.uwaterloo.ca (  0.560 ms  0.559 ms  0.522 ms
 5 (  0.503 ms  0.564 ms  0.566 ms
 6 (  3.160 ms  3.155 ms  3.151 ms
 7 (  5.724 ms  6.534 ms  5.827 ms
 8 (  6.683 ms  6.609 ms  6.681 ms
 9  sunyab-buf-7600.nysernet.net (  7.47 ms  6.906 ms  6.857 ms
10  rtr-internet9-93.cc.buffalo.edu (  7.418 ms  7.131 ms  7.118 ms
11  l3sw-cc9-82.cc.buffalo.edu (  7.557 ms  7.474 ms  7.332 ms
12  vss-cc-bell9-246.cc.buffalo.edu (  8.12 ms  7.779 ms  7.808 ms
13 (  8.187 ms  7.969 ms  7.843 ms
14  moss.cse.buffalo.edu (  7.383 ms  7.430 ms  7.284 ms
hype:tmp {565} sysctl net.inet.tcp.{recv,send}space
hype:tmp {566} uname -rms
OpenBSD 4.6 i386
hype:tmp {567} 


From: (Anonymous)


nyftp.netbsd.org is not a mirror, it is a source (for daily builds and i386+amd64 packages that get built on NetBSD.org owned servers in rackspace donated by Columbia University)

For actual mirrors, you could try if any of ftp://ftp4.us.NetBSD.org/, ftp://ftp5.us.NetBSD.org/ or ftp://ftp7.us.NetBSD.org/ are better for you.
(Ответить) (Thread)
[User Picture]From: cnst

Re: mirrors

I've tried several of these mirrors, and I found that some of them simply didn't work, or those that did work were many times slower than required: ftp4.us is 41ms (umn.edu), ftp5.us is 48ms (iastate.edu) and ftp7.us is 43ms (secsup.org, non-edu), so in the very best case I'd be getting 50Mbps from any of them. In reality, ftp4.us doesn't work (some strange problems with their NcFTPd?), and the speed from ftp5.us and ftp7.us is about 5 to 10 Mbps, which, as I've mentioned, is not acceptable. On the other hand, I still get 100 Mbps from ftp-archive.freebsd.org as previously mentioned.

Could you possibly arrange for some full-blown mirrors in Columbia University? The nyftp source was really fast! :-) That is, until it was taken down, due to the ongoing pkgsrc release process? :/
(Ответить) (Parent) (Thread)
From: (Anonymous)

Re: mirrors

nyftp.n.o was down due to air conditioning maintenance at Columbia, see: http://mail-index.netbsd.org/netbsd-announce/2009/12/23/msg000090.html . It is back now.
(Ответить) (Parent) (Thread)
[User Picture]From: cnst

Re: mirrors

Actually, I think the mirror I've been recently using for some purposes is ftp2.us.NetBSD.org (26ms, non-edu), I get around 75Mbps speeds from it... But it doesn't seem to carry the packages, for example, does it? So right now there's no fast mirror for the packages, since the nyftp source is down.
(Ответить) (Parent) (Thread)
[User Picture]From: cnst

Re: mirrors

FYI: Someone in PR 42547 suggested to adjust kern.sbmax. I adjusted it to 512K, then set recvspace to 384K (12.5MB/s * 30ms = 384K), and now able to get 10.85MiB/s from ftp2.us.NetBSD.org! ftp2.us is 27ms away today. :-)
(Ответить) (Parent) (Thread)
[User Picture]From: cnst

NetBSD PR #42547: "cannot establish connections after setting net.inet.tcp.recvspace sysctl to 256K"

For future reference, and to bring this to the attention of the NetBSD team, I filed a PR regarding the recvspace problem:

(Ответить) (Thread)