Commandline Utilities for Infiniband Network on Exadata

highspeed_data

Many times I need to check out the network traffic on Exadata. 12c OEM Cloud Control is good way to monitor system performance on Exadata. However, sometime I need something quick and want to see the network traffic result from command line. Here are the commands I usually use to check out network traffic for ethernet network and infiniband network.

The first command is dstat.
dstat -dnyc -N eth0,bondeth0,bondib0 -C total -f

Here is the result I run from our X3 box.

Network_dstat_2
The Options I used are
-c enable cpu stats
-d enable disk read/write stats
-n enable network stats (receive, send)
-y enable system stats (interrupts, context switches)

If add three options lms, it will also shows load, memory usage and swap usage.
dstat -dnyclms -N eth0,bondeth0,bondib0 -C total -f

Network_dstat_1

Another command is sar
sar -n DEV 3 100|egrep ‘bondib0|bondeth0|eth0’

network_sar_1

The above command does not show the heading for the sar command. Here the one with heading:

Network_sar_2

In the previous post, iDB vs RDS vs SDP on Exadata, I discussed high level overview about these three different concepts used in Oracle Exadata and related Oracle Engineered Systems. I will show a few more useful commands to illustrate these protocols.

The first command I would like to talk about is ibhost. This InfiniBand command discovers the InfiniBand fabric topology or uses the existing topology file to extract the channel adapter nodes. The followings is the output from our x3 1/8 rack Exadata.

[root@enkx3db01 ~]# ibhosts
Ca	: 0x0010e00b4e20c000 ports 2 "SUN IB QDR GW switch enkbda1sw-ib2 192.168.8.150 Bridge 0"
Ca	: 0x0010e00b4e20c040 ports 2 "SUN IB QDR GW switch enkbda1sw-ib2 192.168.8.150 Bridge 1"
Ca	: 0x0021280001efdf70 ports 2 "enkbda1node10 BDA 192.168.12.110 HCA-1"
Ca	: 0x0021280001efd5ee ports 2 "enkbda1node09 BDA 192.168.12.109 HCA-1"
Ca	: 0x0021280001efd4ea ports 2 "enkbda1node12 BDA 192.168.12.112 HCA-1"
Ca	: 0x0021280001efd4d6 ports 2 "enkbda1node11 BDA 192.168.12.111 HCA-1"
Ca	: 0x0021280001efd5f6 ports 2 "enkbda1node14 BDA 192.168.12.114 HCA-1"
Ca	: 0x0021280001efd4e6 ports 2 "enkbda1node13 BDA 192.168.12.113 HCA-1"
Ca	: 0x0021280001ceda62 ports 2 "enkbda1node16 BDA 192.168.12.116 HCA-1"
Ca	: 0x0021280001cf5abe ports 2 "enkbda1node15 BDA 192.168.12.115 HCA-1"
Ca	: 0x0021280001efac6a ports 2 "enkbda1node18 BDA 192.168.12.118 HCA-1"
Ca	: 0x0021280001efd4fa ports 2 "enkbda1node17 BDA 192.168.12.117 HCA-1"
Ca	: 0x0021280001efdf68 ports 2 "enkbda1node08 BDA 192.168.12.108 HCA-1"
Ca	: 0x0021280001efd5e6 ports 2 "enkbda1node07 BDA 192.168.12.107 HCA-1"
Ca	: 0x0021280001efd606 ports 2 "enkbda1node05 BDA 192.168.12.105 HCA-1"
Ca	: 0x0021280001efd4ee ports 2 "enkbda1node06 BDA 192.168.12.106 HCA-1"
Ca	: 0x0021280001efd616 ports 2 "enkbda1node03 BDA 192.168.12.103 HCA-1"
Ca	: 0x0021280001efdf98 ports 2 "enkbda1node04 BDA 192.168.12.104 HCA-1"
Ca	: 0x0021280001efd84e ports 2 "enkbda1node01 BDA 192.168.12.101 HCA-1"
Ca	: 0x0021280001efdf6c ports 2 "enkbda1node02 BDA 192.168.12.102 HCA-1"
Ca	: 0x0010e00b88c0c000 ports 2 "SUN IB QDR GW switch enkbda1sw-ib3 192.168.8.151 Bridge 0"
Ca	: 0x0010e00b88c0c040 ports 2 "SUN IB QDR GW switch enkbda1sw-ib3 192.168.8.151 Bridge 1"
Ca	: 0x0021280001fcb9ec ports 2 "enkalytics EL-C 192.168.12.131 HCA-1"
Ca	: 0x0021280001fc4a1e ports 2 "enkx3db02 S 192.168.12.2 HCA-1"
Ca	: 0x0021280001fcbf5c ports 2 "enkx3cel03 C 192.168.12.5 HCA-1"
Ca	: 0x0021280001fbe18e ports 2 "enkx3cel01 C 192.168.12.3 HCA-1"
Ca	: 0x0021280001fc80c6 ports 2 "enkx3cel02 C 192.168.12.4 HCA-1"
Ca	: 0x0010e0000128ce64 ports 2 "enkx3db01 S 192.168.12.1 HCA-1"

For a 1/8 rack, it has 2 database nodes, 3 cell nodes and 2 IB switches. You might notice we have many more nodes than supposed to be on the InfiniBand fabric. From the naming, you might figure out we have our X3 Exadata, Oracle Big Data Appliance, and Oracle Exlatics connected together all within the same InfiniBand network.

If just want to list IB switches only, use ibswitches command

[root@enkx3db01 ~]# ibswitches
Switch	: 0x002128f57326a0a0 ports 36 "SUN DCS 36P QDR enkbda1sw-ib1 192.168.8.149" enhanced port 0 lid 59 lmc 0
Switch	: 0x0010e00b88c0c0a0 ports 36 "SUN IB QDR GW switch enkbda1sw-ib3 192.168.8.151" enhanced port 0 lid 61 lmc 0
Switch	: 0x0010e00b4e20c0a0 ports 36 "SUN IB QDR GW switch enkbda1sw-ib2 192.168.8.150" enhanced port 0 lid 60 lmc 0
Switch	: 0x002128f575bba0a0 ports 36 "SUN DCS 36P QDR enkx3sw-ib3.enkitec.com" enhanced port 0 lid 1 lmc 0
Switch	: 0x002128f57469a0a0 ports 36 "SUN DCS 36P QDR enkx3sw-ib2.enkitec.com" enhanced port 0 lid 2 lmc 0

In TCP/IP network, we use ping command to verify whether a host can be accessed or not. Similarly, in InfiniBand network, we use rds-ping command to ping another IB node in the network. The following example shows we could do rds-ping from Exlatics node to the ibvip on the first database node.

[root@enkalytics ~]# rds-ping -c 5 enkx3db01-ibvip.enkitec.com
   1: 240 usec
   2: 214 usec
   3: 201 usec
   4: 199 usec
   5: 269 usec

usec is microseconds.

Another useful rds related command is rds-info. To save space, I removed some smiliar messages.

[root@enkx3db01 ~]# rds-info

RDS IB Connections:
      LocalAddr      RemoteAddr                         LocalDev                        RemoteDev
  192.168.12.31  192.168.12.131           fe80::10:e000:128:ce66           fe80::21:2800:1fc:b9ee
   192.168.12.1    192.168.12.3           fe80::10:e000:128:ce66           fe80::21:2800:1fb:e18f
   192.168.12.1    192.168.12.1           fe80::10:e000:128:ce66           fe80::10:e000:128:ce66
  192.168.12.31   192.168.12.31           fe80::10:e000:128:ce66           fe80::10:e000:128:ce66
   192.168.12.1  192.168.12.101                               ::                               ::
 169.254.87.194  169.254.87.194           fe80::10:e000:128:ce66           fe80::10:e000:128:ce66
   192.168.12.1  192.168.12.118                               ::                               ::
   192.168.12.1    192.168.12.5           fe80::10:e000:128:ce66           fe80::21:2800:1fc:bf5e
   192.168.12.1    192.168.12.4           fe80::10:e000:128:ce66           fe80::21:2800:1fc:80c8
   192.168.12.1    192.168.12.2           fe80::10:e000:128:ce66           fe80::21:2800:1fc:4a20
  192.168.12.31    192.168.12.2           fe80::10:e000:128:ce66           fe80::21:2800:1fc:4a20
 169.254.87.194  169.254.97.245           fe80::10:e000:128:ce66           fe80::21:2800:1fc:4a20
rds-info: Unable get statistics: Protocol not available

Counters:
              CounterName            Value
               conn_reset          2879033
   recv_drop_bad_checksum                0
        recv_drop_old_seq               17
        recv_drop_no_sock             2985
      recv_drop_dead_sock                0
       recv_deliver_raced                0
           recv_delivered        222260977
              recv_queued        130604931
     recv_immediate_retry                0
       recv_delayed_retry                0
        recv_ack_required         14752884
          recv_rdma_bytes     136276672512
                recv_ping           288786
         send_queue_empty         85915668
          send_queue_full               15
     send_lock_contention           764917
    send_lock_queue_raced            16222
     send_immediate_retry                0
       send_delayed_retry             1222
          send_drop_acked                0
        send_ack_required         12818936
              send_queued        115370469
                send_rdma           261202
          send_rdma_bytes     136280842240
                send_pong           288786
       page_remainder_hit        102755106
      page_remainder_miss         11378031
             copy_to_user     165149391125
           copy_from_user     126738139340
       cong_update_queued                0
     cong_update_received               49
          cong_send_error                0
        cong_send_blocked                0
         ib_connect_raced               24
   ib_listen_closed_stale                0
      ib_evt_handler_call        278198961
          ib_tasklet_call        278198961
           ib_tx_cq_event        138358343
          ib_tx_ring_full             1319
           ib_tx_throttle                0
 ib_tx_sg_mapping_failure                0
            ib_tx_stalled              259
     ib_tx_credit_updates                0
           ib_rx_cq_event        172332234
         ib_rx_ring_empty               83
     ib_rx_refill_from_cq                0
 ib_rx_refill_from_thread                0
        ib_rx_alloc_limit                0
     ib_rx_credit_updates                0
              ib_ack_sent         14648252
      ib_ack_send_failure                0
      ib_ack_send_delayed           125532
  ib_ack_send_piggybacked            73902
          ib_ack_received         12960925
         ib_rdma_mr_alloc             6355
          ib_rdma_mr_free             5488
          ib_rdma_mr_used         48938489
    ib_rdma_mr_pool_flush          6438472
     ib_rdma_mr_pool_wait                0
 ib_rdma_mr_pool_depleted                0
           ib_atomic_cswp                0
           ib_atomic_fadd                0
         iw_connect_raced                0
   iw_listen_closed_stale                0
            iw_tx_cq_call                0
           iw_tx_cq_event                0
          iw_tx_ring_full                0
           iw_tx_throttle                0
 iw_tx_sg_mapping_failure                0
            iw_tx_stalled                0
     iw_tx_credit_updates                0
            iw_rx_cq_call                0
           iw_rx_cq_event                0
         iw_rx_ring_empty                0
     iw_rx_refill_from_cq                0
 iw_rx_refill_from_thread                0
        iw_rx_alloc_limit                0
     iw_rx_credit_updates                0
              iw_ack_sent                0
      iw_ack_send_failure                0
      iw_ack_send_delayed                0
  iw_ack_send_piggybacked                0
          iw_ack_received                0
         iw_rdma_mr_alloc                0
          iw_rdma_mr_free                0
          iw_rdma_mr_used                0
    iw_rdma_mr_pool_flush                0
     iw_rdma_mr_pool_wait                0
 iw_rdma_mr_pool_depleted                0

RDS Sockets:
      BoundAddr BPort        ConnAddr CPort     SndBuf     RcvBuf    Inode
   192.168.12.1  7978         0.0.0.0     0     262144    2097152 1422468668
   192.168.12.1 31215         0.0.0.0     0     262144    2097152 1422492506
   192.168.12.1  7588         0.0.0.0     0     262144    2097152 1422492510

....

 169.254.87.194 61081         0.0.0.0     0     131072    2097152 1531223507
   192.168.12.1 16962         0.0.0.0     0     262144    2097152 1534922520
   192.168.12.1   442         0.0.0.0     0     131072    2097152 1534922522
   192.168.12.1 49167         0.0.0.0     0     262144    2097152 1539515072
   192.168.12.1 48917         0.0.0.0     0     131072    2097152 1539515074
   192.168.12.1 14675         0.0.0.0     0     262144    2097152 1539517764
   192.168.12.1 12371         0.0.0.0     0     131072    2097152 1539517766
        0.0.0.0     0         0.0.0.0     0     131072    2097152 1539617228

RDS Connections:
      LocalAddr      RemoteAddr           NextTX           NextRX Flg
  192.168.12.31  192.168.12.131               13               13 --C
   192.168.12.1    192.168.12.3         22473392          1304865 --C
   192.168.12.1    192.168.12.1          2385972           139482 --C
  192.168.12.31   192.168.12.31                9                9 --C
   192.168.12.1  192.168.12.101                4                0 ---
 169.254.87.194  169.254.87.194           492250                0 --C
   192.168.12.1  192.168.12.118              119                0 ---
   192.168.12.1    192.168.12.5         36615339        150138911 --C
   192.168.12.1    192.168.12.4         17227536         60920516 --C
   192.168.12.1    192.168.12.2         28714935          5551186 --C
  192.168.12.31    192.168.12.2              287              287 --C
      127.0.0.1       127.0.0.1            18895            18895 --C
 169.254.87.194  169.254.97.245          7302242          1582910 --C

Receive Message Queue:
      LocalAddr LPort      RemoteAddr RPort              Seq      Bytes
   192.168.12.1 22526    192.168.12.2 20819          3971282        168
   192.168.12.1 22526    192.168.12.2 20819          4210130        168
   192.168.12.1 22526    192.168.12.2 20819          5177334        168
   192.168.12.1 22526    192.168.12.2 20819          5288457        168
   192.168.12.1 44950    192.168.12.2 27716          4485037        168
   192.168.12.1 44950    192.168.12.2 27716          4603330        168
   192.168.12.1 44950    192.168.12.2 27716          4717860        168

....

 169.254.87.194 61209  169.254.97.245  2286          1322929        168
 169.254.87.194 61209  169.254.97.245 32997          1322939        168
   192.168.12.1 62729    192.168.12.2 62458          5513836        168
   192.168.12.1 62729    192.168.12.2 33848          5513844        168
   192.168.12.1 62729    192.168.12.2 37522          5513850        168

Send Message Queue:
      LocalAddr LPort      RemoteAddr RPort              Seq      Bytes

Retransmit Message Queue:
      LocalAddr LPort      RemoteAddr RPort              Seq      Bytes
 169.254.87.194 31175  169.254.87.194 42828           492248        156
 169.254.87.194 31175  169.254.87.194 42828           492249        156
 169.254.87.194   104  169.254.97.245 27039          7302241        252

If you want to test the throughput between two IB nodes, you could use rds-stress.
First, start rds-stress on the target node.

[root@enkx3db01 ~]# rds-stress
waiting for incoming connection on 0.0.0.0:4000

Then, run the following command on source IB node.
rds-stress -s enkx3db01-ibvip.enkitec.com -p 4000 -t 1 -D 600000
-s specify the hostname
-p specify the port number
-t specify the number of tasks
-D specify the total bytes in the RDMA message
After it starts on the source node, the target node will also show the progress.

Source IB node

[root@enkalytics ~]#  rds-stress -s enkx3db01-ibvip.enkitec.com -p 4000 -t 1 -D 600000
connecting to 192.168.12.31:4000
negotiated options, tasks will start in 2 seconds
Starting up....
tsks   tx/s   rx/s  tx+rx K/s    mbi K/s    mbo K/s tx us/c   rtt us cpu %
   1   1668   1668    3531.12  977247.98  977247.98   31.57   558.62 -1.00
   1   1652   1653    3497.93  968354.24  967768.42   34.82   565.02 -1.00
   1   1673   1673    3541.62  980153.86  980153.86   35.05   556.53 -1.00
   1   1682   1682    3560.71  985437.49  985437.49   28.11   555.55 -1.00
   1   1673   1673    3541.71  980179.34  980179.34   29.50   558.08 -1.00
   1   1663   1663    3520.50  974308.84  974308.84   34.43   560.88 -1.00
   1   1692   1692    3581.88  991294.23  991294.23   30.13   552.21 -1.00
   1   1681   1681    3558.60  984852.60  984852.60   29.30   555.98 -1.00
   1   1666   1666    3526.84  976063.53  976063.53   34.09   560.13 -1.00
   1   1678   1678    3552.24  983093.02  983093.02   31.21   556.31 -1.00
   1   1726   1726    3653.88 1011220.94 1011220.94   31.72   538.88 -1.00
   1   1678   1678    3552.26  983097.93  983097.93   29.29   557.05 -1.00
^C

Target IB node

[root@enkx3db01 ~]# rds-stress
waiting for incoming connection on 0.0.0.0:4000
accepted connection from 192.168.12.131:19942 on 192.168.12.31:4000
negotiated options, tasks will start in 2 seconds
Starting up....
tsks   tx/s   rx/s  tx+rx K/s    mbi K/s    mbo K/s tx us/c   rtt us cpu %
   1   1670   1670    3531.99  977489.26  977489.26   15.12   573.84 -1.00
   1   1654   1653    3496.99  967509.78  968095.08   15.71   578.94 -1.00
   1   1675   1675    3542.75  981052.16  979881.45   14.75   570.90 -1.00
   1   1683   1683    3559.70  984572.15  985742.86   15.76   568.26 -1.00
   1   1674   1674    3540.45  979829.57  979829.57   15.93   570.79 -1.00
   1   1666   1666    3523.46  975127.51  975127.51   15.21   575.50 -1.00
   1   1692   1692    3581.02  991057.40  991057.40   16.02   566.28 -1.00
   1   1681   1682    3557.17  984749.23  984163.76   15.50   569.27 -1.00
   1   1667   1667    3526.36  975931.20  975931.20   15.12   574.15 -1.00
   1   1681   1680    3554.11  983903.24  983317.93   15.85   569.99 -1.00
   1   1728   1728    3654.66 1011436.98 1011436.98   16.37   556.93 -1.00
   1   1678   1678    3551.18  982799.19  982799.19   15.53   571.33 -1.00
---------------------------------------------
   1   1677   1677    3551.65  982954.38  982905.59   15.59   570.97 -1.00  (average)

After press CTRL-C on source node, the result average is printed out on target node.

Finally, let’s talk about SDP. My colleague, Andy Colvin, has setup a SDP listener on our X3 Exadata. Here are the command showing Oracle listens on the SDP port.

[enkx3db01:oracle:dbm1] /home/oracle
> srvctl status vip -i enkx3db01-ibvip
VIP enkx3db01-ibvip is enabled
VIP enkx3db01-ibvip is running on node: enkx3db01

[enkx3db01:oracle:dbm1] /home/oracle
> srvctl config listener -l LISTENER_IB
Name: LISTENER_IB
Network: 2, Owner: oracle
Home:
End points: TCP:1522/SDP:1522

[enkx3db01:oracle:+ASM1] /home/oracle
> lsnrctl status LISTENER_IB

LSNRCTL for Linux: Version 11.2.0.3.0 - Production on 09-AUG-2013 21:52:19

Copyright (c) 1991, 2011, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_IB)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER_IB
Version                   TNSLSNR for Linux: Version 11.2.0.3.0 - Production
Start Date                24-JUL-2013 11:36:40
Uptime                    16 days 10 hr. 15 min. 38 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /u01/app/11.2.0.3/grid/network/admin/listener.ora
Listener Log File         /u01/app/11.2.0.3/grid/log/diag/tnslsnr/enkx3db01/listener_ib/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_IB)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=sdp)(HOST=192.168.12.31)(PORT=1522)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.12.31)(PORT=1522)))
Services Summary...
Service "DBM_ETL" has 1 instance(s).
  Instance "dbm1", status READY, has 1 handler(s) for this service...
Service "DBM_REPORTING" has 1 instance(s).
  Instance "dbm1", status READY, has 1 handler(s) for this service...
Service "dbm" has 1 instance(s).
  Instance "dbm1", status READY, has 1 handler(s) for this service...
The command completed successfully

Like netstat command to check out TCP connections, there is corresponding command, sdpnetstat, for SDP connections. Unfortunately, by default, this command does not exist on Exadata for now, but it does exist on Oracle Big Data Appliance or Oracle Exlatics. Here is one example of the output from Exlatics.

[root@enkalytics ~]# sdpnetstat
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 localhost.localdom:6700 localhost.localdo:35791 ESTABLISHED 
tcp        0      0 enkalytics.Enkitec:9710 enkalytics.Enkite:42991 ESTABLISHED 
tcp        0      0 enkalytics.Enkitec:9706 enkalytics.Enkite:11093 ESTABLISHED 
tcp        0      0 enkalytics.Enkitec:9701 enkalytics.Enkite:16776 ESTABLISHED 
tcp        0      0 localhost.localdom:6700 localhost.localdo:35793 ESTABLISHED 
tcp        0      0 enkalytics.Enkitec:9710 enkalytics.Enkite:10002 ESTABLISHED 
tcp        0      0 localhost.localdom:6700 localhost.localdo:35790 ESTABLISHED 
....
....
tcp        0      0 localhost.localdo:35791 localhost.localdom:6700 ESTABLISHED 
tcp        0      0 localhost.localdom:6700 localhost.localdo:35798 ESTABLISHED 
tcp        0      0 enkalytics.Enkite:11093 enkalytics.Enkitec:9706 ESTABLISHED 
tcp        0      0 localhost.localdom:6700 localhost.localdo:35797 ESTABLISHED 
tcp        0      0 localhost.localdo:35793 localhost.localdom:6700 ESTABLISHED 
tcp        0      0 enkalytics.Enkite:60565 enkalytics.Enkitec:9710 TIME_WAIT   
tcp        0      0 enkalytics.enkite:36136 enk03-vip.enki:ncube-lm ESTABLISHED 
tcp        0      0 enkalytics.enkite:21666 enkalytic:afs3-callback ESTABLISHED 
tcp        0      0 enkalytics.enkitec:9704 enkalytics.enkite:52854 TIME_WAIT   
tcp        0      0 enkalytics.enkitec:9704 enkalytics.enkite:52849 TIME_WAIT   
tcp        0      0 enkalytics.enkite:25478 enk04-vip.enki:ncube-lm ESTABLISHED 
tcp        0      0 enkalytic:afs3-callback enkalytics.enkite:11226 ESTABLISHED 
tcp        0      0 enkalytics.enkite:44861 enkalytics.enkitec:9704 ESTABLISHED 
tcp        0      0 enkalytics.enkitec:9704 enkalytics.enkite:52850 TIME_WAIT   
tcp        0      0 enkalytics.enkite:44867 enkalytics.enkitec:9704 ESTABLISHED 
tcp        0      0 enkalytics.enkitec:9704 enkalytics.enkite:52846 TIME_WAIT   
tcp        0      0 enkalytics.enkitec:9704 enkalytics.enkite:44867 ESTABLISHED 
tcp        0      0 localhost.localdo:35797 localhost.localdom:6700 ESTABLISHED 
tcp        0      0 enkalytics.enkite:11226 enkalytic:afs3-callback ESTABLISHED 
tcp        0      0 enkalytics.enkitec:9704 enkalytics.enkite:44861 ESTABLISHED 
tcp        0      0 enkalytic:afs3-callback enkalytics.enkite:11210 ESTABLISHED 
sdp        0      0 192.168.12.131:43307    enkx3db01-ib:ricardo-lm ESTABLISHED 
Active UNIX domain sockets (w/o servers)
Proto RefCnt Flags       Type       State         I-Node Path
unix  20     [ ]         DGRAM                    24350  /dev/log
unix  2      [ ]         DGRAM                    7979   @/org/kernel/udev/udevd
unix  2      [ ]         DGRAM                    27732  @/org/freedesktop/hal/udev_event
unix  2      [ ]         DGRAM                    4592850 
unix  2      [ ]         STREAM     CONNECTED     1138317 
unix  2      [ ]         STREAM     CONNECTED     1104247 
unix  2      [ ]         STREAM     CONNECTED     1099975 
unix  2      [ ]         STREAM     CONNECTED     1099714 
unix  3      [ ]         STREAM     CONNECTED     971435 /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     971434 
unix  2      [ ]         DGRAM                    478947 
unix  3      [ ]         STREAM     CONNECTED     29367  @/tmp/fam-root-
unix  3      [ ]         STREAM     CONNECTED     29366  
unix  3      [ ]         STREAM     CONNECTED     29353  /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     29352  
unix  3      [ ]         STREAM     CONNECTED     29170  /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     29169  
unix  3      [ ]         STREAM     CONNECTED     29164  
unix  3      [ ]         STREAM     CONNECTED     29163  
unix  2      [ ]         DGRAM                    29161  
unix  2      [ ]         DGRAM                    28889  
....
....
unix  2      [ ]         DGRAM                    27890  
unix  3      [ ]         STREAM     CONNECTED     27865  /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     27864  
unix  3      [ ]         STREAM     CONNECTED     27813  @/var/run/hald/dbus-aeLDAYiwqS
unix  3      [ ]         STREAM     CONNECTED     27812  
unix  3      [ ]         STREAM     CONNECTED     27798  @/var/run/hald/dbus-aeLDAYiwqS
unix  3      [ ]         STREAM     CONNECTED     27797  
unix  3      [ ]         STREAM     CONNECTED     27783  @/var/run/hald/dbus-aeLDAYiwqS
unix  3      [ ]         STREAM     CONNECTED     27782  
unix  3      [ ]         STREAM     CONNECTED     27766  /var/run/acpid.socket
unix  3      [ ]         STREAM     CONNECTED     27765  
unix  3      [ ]         STREAM     CONNECTED     27760  @/var/run/hald/dbus-aeLDAYiwqS
unix  3      [ ]         STREAM     CONNECTED     27759  
unix  3      [ ]         STREAM     CONNECTED     27727  @/var/run/hald/dbus-0e5V2Tfgxi
unix  3      [ ]         STREAM     CONNECTED     27726  
unix  2      [ ]         DGRAM                    27562  
unix  3      [ ]         STREAM     CONNECTED     27445  /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     27444  
unix  2      [ ]         DGRAM                    27433  
unix  2      [ ]         DGRAM                    27422  
unix  3      [ ]         STREAM     CONNECTED     27381  
unix  3      [ ]         STREAM     CONNECTED     27380  
unix  3      [ ]         STREAM     CONNECTED     27339  
unix  3      [ ]         STREAM     CONNECTED     27338  
unix  2      [ ]         DGRAM                    26918  
unix  2      [ ]         DGRAM                    24358  
unix  3      [ ]         STREAM     CONNECTED     24299  
unix  3      [ ]         STREAM     CONNECTED     24298  

[root@enkalytics ~]# ifconfig 
bond0     Link encap:InfiniBand  HWaddr 80:00:00:4B:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00  
          inet addr:192.168.12.131  Bcast:192.168.12.255  Mask:255.255.255.0
          inet6 addr: fe80::221:2800:1fc:b9ed/64 Scope:Link
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:65520  Metric:1
          RX packets:102393 errors:0 dropped:0 overruns:0 frame:0
          TX packets:133607 errors:0 dropped:16 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:6058691 (5.7 MiB)  TX bytes:4830507 (4.6 MiB)
....

Advertisements

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s