Many times I need to check out the network traffic on Exadata. 12c OEM Cloud Control is good way to monitor system performance on Exadata. However, sometime I need something quick and want to see the network traffic result from command line. Here are the commands I usually use to check out network traffic for ethernet network and infiniband network.
The first command is dstat.
dstat -dnyc -N eth0,bondeth0,bondib0 -C total -f
Here is the result I run from our X3 box.
The Options I used are
-c enable cpu stats
-d enable disk read/write stats
-n enable network stats (receive, send)
-y enable system stats (interrupts, context switches)
If add three options lms, it will also shows load, memory usage and swap usage.
dstat -dnyclms -N eth0,bondeth0,bondib0 -C total -f
Another command is sar
sar -n DEV 3 100|egrep ‘bondib0|bondeth0|eth0’
The above command does not show the heading for the sar command. Here the one with heading:
In the previous post, iDB vs RDS vs SDP on Exadata, I discussed high level overview about these three different concepts used in Oracle Exadata and related Oracle Engineered Systems. I will show a few more useful commands to illustrate these protocols.
The first command I would like to talk about is ibhost. This InfiniBand command discovers the InfiniBand fabric topology or uses the existing topology file to extract the channel adapter nodes. The followings is the output from our x3 1/8 rack Exadata.
[root@enkx3db01 ~]# ibhosts Ca : 0x0010e00b4e20c000 ports 2 "SUN IB QDR GW switch enkbda1sw-ib2 192.168.8.150 Bridge 0" Ca : 0x0010e00b4e20c040 ports 2 "SUN IB QDR GW switch enkbda1sw-ib2 192.168.8.150 Bridge 1" Ca : 0x0021280001efdf70 ports 2 "enkbda1node10 BDA 192.168.12.110 HCA-1" Ca : 0x0021280001efd5ee ports 2 "enkbda1node09 BDA 192.168.12.109 HCA-1" Ca : 0x0021280001efd4ea ports 2 "enkbda1node12 BDA 192.168.12.112 HCA-1" Ca : 0x0021280001efd4d6 ports 2 "enkbda1node11 BDA 192.168.12.111 HCA-1" Ca : 0x0021280001efd5f6 ports 2 "enkbda1node14 BDA 192.168.12.114 HCA-1" Ca : 0x0021280001efd4e6 ports 2 "enkbda1node13 BDA 192.168.12.113 HCA-1" Ca : 0x0021280001ceda62 ports 2 "enkbda1node16 BDA 192.168.12.116 HCA-1" Ca : 0x0021280001cf5abe ports 2 "enkbda1node15 BDA 192.168.12.115 HCA-1" Ca : 0x0021280001efac6a ports 2 "enkbda1node18 BDA 192.168.12.118 HCA-1" Ca : 0x0021280001efd4fa ports 2 "enkbda1node17 BDA 192.168.12.117 HCA-1" Ca : 0x0021280001efdf68 ports 2 "enkbda1node08 BDA 192.168.12.108 HCA-1" Ca : 0x0021280001efd5e6 ports 2 "enkbda1node07 BDA 192.168.12.107 HCA-1" Ca : 0x0021280001efd606 ports 2 "enkbda1node05 BDA 192.168.12.105 HCA-1" Ca : 0x0021280001efd4ee ports 2 "enkbda1node06 BDA 192.168.12.106 HCA-1" Ca : 0x0021280001efd616 ports 2 "enkbda1node03 BDA 192.168.12.103 HCA-1" Ca : 0x0021280001efdf98 ports 2 "enkbda1node04 BDA 192.168.12.104 HCA-1" Ca : 0x0021280001efd84e ports 2 "enkbda1node01 BDA 192.168.12.101 HCA-1" Ca : 0x0021280001efdf6c ports 2 "enkbda1node02 BDA 192.168.12.102 HCA-1" Ca : 0x0010e00b88c0c000 ports 2 "SUN IB QDR GW switch enkbda1sw-ib3 192.168.8.151 Bridge 0" Ca : 0x0010e00b88c0c040 ports 2 "SUN IB QDR GW switch enkbda1sw-ib3 192.168.8.151 Bridge 1" Ca : 0x0021280001fcb9ec ports 2 "enkalytics EL-C 192.168.12.131 HCA-1" Ca : 0x0021280001fc4a1e ports 2 "enkx3db02 S 192.168.12.2 HCA-1" Ca : 0x0021280001fcbf5c ports 2 "enkx3cel03 C 192.168.12.5 HCA-1" Ca : 0x0021280001fbe18e ports 2 "enkx3cel01 C 192.168.12.3 HCA-1" Ca : 0x0021280001fc80c6 ports 2 "enkx3cel02 C 192.168.12.4 HCA-1" Ca : 0x0010e0000128ce64 ports 2 "enkx3db01 S 192.168.12.1 HCA-1"
For a 1/8 rack, it has 2 database nodes, 3 cell nodes and 2 IB switches. You might notice we have many more nodes than supposed to be on the InfiniBand fabric. From the naming, you might figure out we have our X3 Exadata, Oracle Big Data Appliance, and Oracle Exlatics connected together all within the same InfiniBand network.
If just want to list IB switches only, use ibswitches command
[root@enkx3db01 ~]# ibswitches Switch : 0x002128f57326a0a0 ports 36 "SUN DCS 36P QDR enkbda1sw-ib1 192.168.8.149" enhanced port 0 lid 59 lmc 0 Switch : 0x0010e00b88c0c0a0 ports 36 "SUN IB QDR GW switch enkbda1sw-ib3 192.168.8.151" enhanced port 0 lid 61 lmc 0 Switch : 0x0010e00b4e20c0a0 ports 36 "SUN IB QDR GW switch enkbda1sw-ib2 192.168.8.150" enhanced port 0 lid 60 lmc 0 Switch : 0x002128f575bba0a0 ports 36 "SUN DCS 36P QDR enkx3sw-ib3.enkitec.com" enhanced port 0 lid 1 lmc 0 Switch : 0x002128f57469a0a0 ports 36 "SUN DCS 36P QDR enkx3sw-ib2.enkitec.com" enhanced port 0 lid 2 lmc 0
In TCP/IP network, we use ping command to verify whether a host can be accessed or not. Similarly, in InfiniBand network, we use rds-ping command to ping another IB node in the network. The following example shows we could do rds-ping from Exlatics node to the ibvip on the first database node.
[root@enkalytics ~]# rds-ping -c 5 enkx3db01-ibvip.enkitec.com 1: 240 usec 2: 214 usec 3: 201 usec 4: 199 usec 5: 269 usec
usec is microseconds.
Another useful rds related command is rds-info. To save space, I removed some smiliar messages.
[root@enkx3db01 ~]# rds-info
RDS IB Connections:
LocalAddr RemoteAddr LocalDev RemoteDev
192.168.12.31 192.168.12.131 fe80::10:e000:128:ce66 fe80::21:2800:1fc:b9ee
192.168.12.1 192.168.12.3 fe80::10:e000:128:ce66 fe80::21:2800:1fb:e18f
192.168.12.1 192.168.12.1 fe80::10:e000:128:ce66 fe80::10:e000:128:ce66
192.168.12.31 192.168.12.31 fe80::10:e000:128:ce66 fe80::10:e000:128:ce66
192.168.12.1 192.168.12.101 :: ::
169.254.87.194 169.254.87.194 fe80::10:e000:128:ce66 fe80::10:e000:128:ce66
192.168.12.1 192.168.12.118 :: ::
192.168.12.1 192.168.12.5 fe80::10:e000:128:ce66 fe80::21:2800:1fc:bf5e
192.168.12.1 192.168.12.4 fe80::10:e000:128:ce66 fe80::21:2800:1fc:80c8
192.168.12.1 192.168.12.2 fe80::10:e000:128:ce66 fe80::21:2800:1fc:4a20
192.168.12.31 192.168.12.2 fe80::10:e000:128:ce66 fe80::21:2800:1fc:4a20
169.254.87.194 169.254.97.245 fe80::10:e000:128:ce66 fe80::21:2800:1fc:4a20
rds-info: Unable get statistics: Protocol not available
Counters:
CounterName Value
conn_reset 2879033
recv_drop_bad_checksum 0
recv_drop_old_seq 17
recv_drop_no_sock 2985
recv_drop_dead_sock 0
recv_deliver_raced 0
recv_delivered 222260977
recv_queued 130604931
recv_immediate_retry 0
recv_delayed_retry 0
recv_ack_required 14752884
recv_rdma_bytes 136276672512
recv_ping 288786
send_queue_empty 85915668
send_queue_full 15
send_lock_contention 764917
send_lock_queue_raced 16222
send_immediate_retry 0
send_delayed_retry 1222
send_drop_acked 0
send_ack_required 12818936
send_queued 115370469
send_rdma 261202
send_rdma_bytes 136280842240
send_pong 288786
page_remainder_hit 102755106
page_remainder_miss 11378031
copy_to_user 165149391125
copy_from_user 126738139340
cong_update_queued 0
cong_update_received 49
cong_send_error 0
cong_send_blocked 0
ib_connect_raced 24
ib_listen_closed_stale 0
ib_evt_handler_call 278198961
ib_tasklet_call 278198961
ib_tx_cq_event 138358343
ib_tx_ring_full 1319
ib_tx_throttle 0
ib_tx_sg_mapping_failure 0
ib_tx_stalled 259
ib_tx_credit_updates 0
ib_rx_cq_event 172332234
ib_rx_ring_empty 83
ib_rx_refill_from_cq 0
ib_rx_refill_from_thread 0
ib_rx_alloc_limit 0
ib_rx_credit_updates 0
ib_ack_sent 14648252
ib_ack_send_failure 0
ib_ack_send_delayed 125532
ib_ack_send_piggybacked 73902
ib_ack_received 12960925
ib_rdma_mr_alloc 6355
ib_rdma_mr_free 5488
ib_rdma_mr_used 48938489
ib_rdma_mr_pool_flush 6438472
ib_rdma_mr_pool_wait 0
ib_rdma_mr_pool_depleted 0
ib_atomic_cswp 0
ib_atomic_fadd 0
iw_connect_raced 0
iw_listen_closed_stale 0
iw_tx_cq_call 0
iw_tx_cq_event 0
iw_tx_ring_full 0
iw_tx_throttle 0
iw_tx_sg_mapping_failure 0
iw_tx_stalled 0
iw_tx_credit_updates 0
iw_rx_cq_call 0
iw_rx_cq_event 0
iw_rx_ring_empty 0
iw_rx_refill_from_cq 0
iw_rx_refill_from_thread 0
iw_rx_alloc_limit 0
iw_rx_credit_updates 0
iw_ack_sent 0
iw_ack_send_failure 0
iw_ack_send_delayed 0
iw_ack_send_piggybacked 0
iw_ack_received 0
iw_rdma_mr_alloc 0
iw_rdma_mr_free 0
iw_rdma_mr_used 0
iw_rdma_mr_pool_flush 0
iw_rdma_mr_pool_wait 0
iw_rdma_mr_pool_depleted 0
RDS Sockets:
BoundAddr BPort ConnAddr CPort SndBuf RcvBuf Inode
192.168.12.1 7978 0.0.0.0 0 262144 2097152 1422468668
192.168.12.1 31215 0.0.0.0 0 262144 2097152 1422492506
192.168.12.1 7588 0.0.0.0 0 262144 2097152 1422492510
....
169.254.87.194 61081 0.0.0.0 0 131072 2097152 1531223507
192.168.12.1 16962 0.0.0.0 0 262144 2097152 1534922520
192.168.12.1 442 0.0.0.0 0 131072 2097152 1534922522
192.168.12.1 49167 0.0.0.0 0 262144 2097152 1539515072
192.168.12.1 48917 0.0.0.0 0 131072 2097152 1539515074
192.168.12.1 14675 0.0.0.0 0 262144 2097152 1539517764
192.168.12.1 12371 0.0.0.0 0 131072 2097152 1539517766
0.0.0.0 0 0.0.0.0 0 131072 2097152 1539617228
RDS Connections:
LocalAddr RemoteAddr NextTX NextRX Flg
192.168.12.31 192.168.12.131 13 13 --C
192.168.12.1 192.168.12.3 22473392 1304865 --C
192.168.12.1 192.168.12.1 2385972 139482 --C
192.168.12.31 192.168.12.31 9 9 --C
192.168.12.1 192.168.12.101 4 0 ---
169.254.87.194 169.254.87.194 492250 0 --C
192.168.12.1 192.168.12.118 119 0 ---
192.168.12.1 192.168.12.5 36615339 150138911 --C
192.168.12.1 192.168.12.4 17227536 60920516 --C
192.168.12.1 192.168.12.2 28714935 5551186 --C
192.168.12.31 192.168.12.2 287 287 --C
127.0.0.1 127.0.0.1 18895 18895 --C
169.254.87.194 169.254.97.245 7302242 1582910 --C
Receive Message Queue:
LocalAddr LPort RemoteAddr RPort Seq Bytes
192.168.12.1 22526 192.168.12.2 20819 3971282 168
192.168.12.1 22526 192.168.12.2 20819 4210130 168
192.168.12.1 22526 192.168.12.2 20819 5177334 168
192.168.12.1 22526 192.168.12.2 20819 5288457 168
192.168.12.1 44950 192.168.12.2 27716 4485037 168
192.168.12.1 44950 192.168.12.2 27716 4603330 168
192.168.12.1 44950 192.168.12.2 27716 4717860 168
....
169.254.87.194 61209 169.254.97.245 2286 1322929 168
169.254.87.194 61209 169.254.97.245 32997 1322939 168
192.168.12.1 62729 192.168.12.2 62458 5513836 168
192.168.12.1 62729 192.168.12.2 33848 5513844 168
192.168.12.1 62729 192.168.12.2 37522 5513850 168
Send Message Queue:
LocalAddr LPort RemoteAddr RPort Seq Bytes
Retransmit Message Queue:
LocalAddr LPort RemoteAddr RPort Seq Bytes
169.254.87.194 31175 169.254.87.194 42828 492248 156
169.254.87.194 31175 169.254.87.194 42828 492249 156
169.254.87.194 104 169.254.97.245 27039 7302241 252
If you want to test the throughput between two IB nodes, you could use rds-stress.
First, start rds-stress on the target node.
[root@enkx3db01 ~]# rds-stress waiting for incoming connection on 0.0.0.0:4000
Then, run the following command on source IB node.
rds-stress -s enkx3db01-ibvip.enkitec.com -p 4000 -t 1 -D 600000
-s specify the hostname
-p specify the port number
-t specify the number of tasks
-D specify the total bytes in the RDMA message
After it starts on the source node, the target node will also show the progress.
Source IB node
[root@enkalytics ~]# rds-stress -s enkx3db01-ibvip.enkitec.com -p 4000 -t 1 -D 600000 connecting to 192.168.12.31:4000 negotiated options, tasks will start in 2 seconds Starting up.... tsks tx/s rx/s tx+rx K/s mbi K/s mbo K/s tx us/c rtt us cpu % 1 1668 1668 3531.12 977247.98 977247.98 31.57 558.62 -1.00 1 1652 1653 3497.93 968354.24 967768.42 34.82 565.02 -1.00 1 1673 1673 3541.62 980153.86 980153.86 35.05 556.53 -1.00 1 1682 1682 3560.71 985437.49 985437.49 28.11 555.55 -1.00 1 1673 1673 3541.71 980179.34 980179.34 29.50 558.08 -1.00 1 1663 1663 3520.50 974308.84 974308.84 34.43 560.88 -1.00 1 1692 1692 3581.88 991294.23 991294.23 30.13 552.21 -1.00 1 1681 1681 3558.60 984852.60 984852.60 29.30 555.98 -1.00 1 1666 1666 3526.84 976063.53 976063.53 34.09 560.13 -1.00 1 1678 1678 3552.24 983093.02 983093.02 31.21 556.31 -1.00 1 1726 1726 3653.88 1011220.94 1011220.94 31.72 538.88 -1.00 1 1678 1678 3552.26 983097.93 983097.93 29.29 557.05 -1.00 ^C
Target IB node
[root@enkx3db01 ~]# rds-stress waiting for incoming connection on 0.0.0.0:4000 accepted connection from 192.168.12.131:19942 on 192.168.12.31:4000 negotiated options, tasks will start in 2 seconds Starting up.... tsks tx/s rx/s tx+rx K/s mbi K/s mbo K/s tx us/c rtt us cpu % 1 1670 1670 3531.99 977489.26 977489.26 15.12 573.84 -1.00 1 1654 1653 3496.99 967509.78 968095.08 15.71 578.94 -1.00 1 1675 1675 3542.75 981052.16 979881.45 14.75 570.90 -1.00 1 1683 1683 3559.70 984572.15 985742.86 15.76 568.26 -1.00 1 1674 1674 3540.45 979829.57 979829.57 15.93 570.79 -1.00 1 1666 1666 3523.46 975127.51 975127.51 15.21 575.50 -1.00 1 1692 1692 3581.02 991057.40 991057.40 16.02 566.28 -1.00 1 1681 1682 3557.17 984749.23 984163.76 15.50 569.27 -1.00 1 1667 1667 3526.36 975931.20 975931.20 15.12 574.15 -1.00 1 1681 1680 3554.11 983903.24 983317.93 15.85 569.99 -1.00 1 1728 1728 3654.66 1011436.98 1011436.98 16.37 556.93 -1.00 1 1678 1678 3551.18 982799.19 982799.19 15.53 571.33 -1.00 --------------------------------------------- 1 1677 1677 3551.65 982954.38 982905.59 15.59 570.97 -1.00 (average)
After press CTRL-C on source node, the result average is printed out on target node.
Finally, let’s talk about SDP. My colleague, Andy Colvin, has setup a SDP listener on our X3 Exadata. Here are the command showing Oracle listens on the SDP port.
[enkx3db01:oracle:dbm1] /home/oracle > srvctl status vip -i enkx3db01-ibvip VIP enkx3db01-ibvip is enabled VIP enkx3db01-ibvip is running on node: enkx3db01 [enkx3db01:oracle:dbm1] /home/oracle > srvctl config listener -l LISTENER_IB Name: LISTENER_IB Network: 2, Owner: oracle Home: End points: TCP:1522/SDP:1522 [enkx3db01:oracle:+ASM1] /home/oracle > lsnrctl status LISTENER_IB LSNRCTL for Linux: Version 11.2.0.3.0 - Production on 09-AUG-2013 21:52:19 Copyright (c) 1991, 2011, Oracle. All rights reserved. Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_IB))) STATUS of the LISTENER ------------------------ Alias LISTENER_IB Version TNSLSNR for Linux: Version 11.2.0.3.0 - Production Start Date 24-JUL-2013 11:36:40 Uptime 16 days 10 hr. 15 min. 38 sec Trace Level off Security ON: Local OS Authentication SNMP OFF Listener Parameter File /u01/app/11.2.0.3/grid/network/admin/listener.ora Listener Log File /u01/app/11.2.0.3/grid/log/diag/tnslsnr/enkx3db01/listener_ib/alert/log.xml Listening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_IB))) (DESCRIPTION=(ADDRESS=(PROTOCOL=sdp)(HOST=192.168.12.31)(PORT=1522))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.12.31)(PORT=1522))) Services Summary... Service "DBM_ETL" has 1 instance(s). Instance "dbm1", status READY, has 1 handler(s) for this service... Service "DBM_REPORTING" has 1 instance(s). Instance "dbm1", status READY, has 1 handler(s) for this service... Service "dbm" has 1 instance(s). Instance "dbm1", status READY, has 1 handler(s) for this service... The command completed successfully
Like netstat command to check out TCP connections, there is corresponding command, sdpnetstat, for SDP connections. Unfortunately, by default, this command does not exist on Exadata for now, but it does exist on Oracle Big Data Appliance or Oracle Exlatics. Here is one example of the output from Exlatics.
[root@enkalytics ~]# sdpnetstat
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 localhost.localdom:6700 localhost.localdo:35791 ESTABLISHED
tcp 0 0 enkalytics.Enkitec:9710 enkalytics.Enkite:42991 ESTABLISHED
tcp 0 0 enkalytics.Enkitec:9706 enkalytics.Enkite:11093 ESTABLISHED
tcp 0 0 enkalytics.Enkitec:9701 enkalytics.Enkite:16776 ESTABLISHED
tcp 0 0 localhost.localdom:6700 localhost.localdo:35793 ESTABLISHED
tcp 0 0 enkalytics.Enkitec:9710 enkalytics.Enkite:10002 ESTABLISHED
tcp 0 0 localhost.localdom:6700 localhost.localdo:35790 ESTABLISHED
....
....
tcp 0 0 localhost.localdo:35791 localhost.localdom:6700 ESTABLISHED
tcp 0 0 localhost.localdom:6700 localhost.localdo:35798 ESTABLISHED
tcp 0 0 enkalytics.Enkite:11093 enkalytics.Enkitec:9706 ESTABLISHED
tcp 0 0 localhost.localdom:6700 localhost.localdo:35797 ESTABLISHED
tcp 0 0 localhost.localdo:35793 localhost.localdom:6700 ESTABLISHED
tcp 0 0 enkalytics.Enkite:60565 enkalytics.Enkitec:9710 TIME_WAIT
tcp 0 0 enkalytics.enkite:36136 enk03-vip.enki:ncube-lm ESTABLISHED
tcp 0 0 enkalytics.enkite:21666 enkalytic:afs3-callback ESTABLISHED
tcp 0 0 enkalytics.enkitec:9704 enkalytics.enkite:52854 TIME_WAIT
tcp 0 0 enkalytics.enkitec:9704 enkalytics.enkite:52849 TIME_WAIT
tcp 0 0 enkalytics.enkite:25478 enk04-vip.enki:ncube-lm ESTABLISHED
tcp 0 0 enkalytic:afs3-callback enkalytics.enkite:11226 ESTABLISHED
tcp 0 0 enkalytics.enkite:44861 enkalytics.enkitec:9704 ESTABLISHED
tcp 0 0 enkalytics.enkitec:9704 enkalytics.enkite:52850 TIME_WAIT
tcp 0 0 enkalytics.enkite:44867 enkalytics.enkitec:9704 ESTABLISHED
tcp 0 0 enkalytics.enkitec:9704 enkalytics.enkite:52846 TIME_WAIT
tcp 0 0 enkalytics.enkitec:9704 enkalytics.enkite:44867 ESTABLISHED
tcp 0 0 localhost.localdo:35797 localhost.localdom:6700 ESTABLISHED
tcp 0 0 enkalytics.enkite:11226 enkalytic:afs3-callback ESTABLISHED
tcp 0 0 enkalytics.enkitec:9704 enkalytics.enkite:44861 ESTABLISHED
tcp 0 0 enkalytic:afs3-callback enkalytics.enkite:11210 ESTABLISHED
sdp 0 0 192.168.12.131:43307 enkx3db01-ib:ricardo-lm ESTABLISHED
Active UNIX domain sockets (w/o servers)
Proto RefCnt Flags Type State I-Node Path
unix 20 [ ] DGRAM 24350 /dev/log
unix 2 [ ] DGRAM 7979 @/org/kernel/udev/udevd
unix 2 [ ] DGRAM 27732 @/org/freedesktop/hal/udev_event
unix 2 [ ] DGRAM 4592850
unix 2 [ ] STREAM CONNECTED 1138317
unix 2 [ ] STREAM CONNECTED 1104247
unix 2 [ ] STREAM CONNECTED 1099975
unix 2 [ ] STREAM CONNECTED 1099714
unix 3 [ ] STREAM CONNECTED 971435 /var/run/dbus/system_bus_socket
unix 3 [ ] STREAM CONNECTED 971434
unix 2 [ ] DGRAM 478947
unix 3 [ ] STREAM CONNECTED 29367 @/tmp/fam-root-
unix 3 [ ] STREAM CONNECTED 29366
unix 3 [ ] STREAM CONNECTED 29353 /var/run/dbus/system_bus_socket
unix 3 [ ] STREAM CONNECTED 29352
unix 3 [ ] STREAM CONNECTED 29170 /var/run/dbus/system_bus_socket
unix 3 [ ] STREAM CONNECTED 29169
unix 3 [ ] STREAM CONNECTED 29164
unix 3 [ ] STREAM CONNECTED 29163
unix 2 [ ] DGRAM 29161
unix 2 [ ] DGRAM 28889
....
....
unix 2 [ ] DGRAM 27890
unix 3 [ ] STREAM CONNECTED 27865 /var/run/dbus/system_bus_socket
unix 3 [ ] STREAM CONNECTED 27864
unix 3 [ ] STREAM CONNECTED 27813 @/var/run/hald/dbus-aeLDAYiwqS
unix 3 [ ] STREAM CONNECTED 27812
unix 3 [ ] STREAM CONNECTED 27798 @/var/run/hald/dbus-aeLDAYiwqS
unix 3 [ ] STREAM CONNECTED 27797
unix 3 [ ] STREAM CONNECTED 27783 @/var/run/hald/dbus-aeLDAYiwqS
unix 3 [ ] STREAM CONNECTED 27782
unix 3 [ ] STREAM CONNECTED 27766 /var/run/acpid.socket
unix 3 [ ] STREAM CONNECTED 27765
unix 3 [ ] STREAM CONNECTED 27760 @/var/run/hald/dbus-aeLDAYiwqS
unix 3 [ ] STREAM CONNECTED 27759
unix 3 [ ] STREAM CONNECTED 27727 @/var/run/hald/dbus-0e5V2Tfgxi
unix 3 [ ] STREAM CONNECTED 27726
unix 2 [ ] DGRAM 27562
unix 3 [ ] STREAM CONNECTED 27445 /var/run/dbus/system_bus_socket
unix 3 [ ] STREAM CONNECTED 27444
unix 2 [ ] DGRAM 27433
unix 2 [ ] DGRAM 27422
unix 3 [ ] STREAM CONNECTED 27381
unix 3 [ ] STREAM CONNECTED 27380
unix 3 [ ] STREAM CONNECTED 27339
unix 3 [ ] STREAM CONNECTED 27338
unix 2 [ ] DGRAM 26918
unix 2 [ ] DGRAM 24358
unix 3 [ ] STREAM CONNECTED 24299
unix 3 [ ] STREAM CONNECTED 24298
[root@enkalytics ~]# ifconfig
bond0 Link encap:InfiniBand HWaddr 80:00:00:4B:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00
inet addr:192.168.12.131 Bcast:192.168.12.255 Mask:255.255.255.0
inet6 addr: fe80::221:2800:1fc:b9ed/64 Scope:Link
UP BROADCAST RUNNING MASTER MULTICAST MTU:65520 Metric:1
RX packets:102393 errors:0 dropped:0 overruns:0 frame:0
TX packets:133607 errors:0 dropped:16 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:6058691 (5.7 MiB) TX bytes:4830507 (4.6 MiB)
....
