netstat and ss - Commands

Shared mappers for parsing and extracting data from variations of the netstat and ss commands. Mappers contained in this module are:

NetstatS - command netstat -s

NetstatAGN - command netstat -agn

Netstat - command netstat -neopa

Netstat_I - command netstat -i

SsTULPN - command ss -tulpn

SsTUPNA - command ss -tupna

ProcNsat - File /proc/net/netstat

insights.parsers.netstat.ACTIVE_INTERNET_CONNECTIONS = 'Active Internet connections (servers and established)'

The key in Netstat data to internet connection information

Type:

str

insights.parsers.netstat.ACTIVE_UNIX_DOMAIN_SOCKETS = 'Active UNIX domain sockets (servers and established)'

The key in Netstat data UNIX domain socket information

Type:

str

class insights.parsers.netstat.Netstat(context, extra_bad_lines=None)[source]

Bases: CommandParser

Parsing the /bin/netstat -neopa command output.

Example output:

Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       User       Inode      PID/Program name     Timer
tcp        0      0 0.0.0.0:5672            0.0.0.0:*               LISTEN      996        19422      1279/qpidd           off (0.00/0/0)
tcp        0      0 127.0.0.1:27017         0.0.0.0:*               LISTEN      184        20380      2007/mongod          off (0.00/0/0)
tcp        0      0 127.0.0.1:53644         0.0.0.0:*               LISTEN      995        1154674    12387/Passenger Rac  off (0.00/0/0)
tcp        0      0 0.0.0.0:5646            0.0.0.0:*               LISTEN      991        20182      1272/qdrouterd       off (0.00/0/0)
Active UNIX domain sockets (servers and established)
Proto RefCnt Flags       Type       State         I-Node   PID/Program name     Path
unix  2      [ ]         DGRAM                    11776    1/systemd            /run/systemd/shutdownd
unix  2      [ ACC ]     STREAM     LISTENING     535      1/systemd            /run/lvm/lvmetad.socket
unix  2      [ ACC ]     STREAM     LISTENING     16411    738/NetworkManager   /var/run/NetworkManager/private

The following attributes are all keyed on the header as it appears complete in the input - e.g. active connections are stored by the key ‘Active Internet connections (servers and established)’. For convenience, these two keys are stored in this module under the constant names:

  • ACTIVE_INTERNET_CONNECTIONS

  • ACTIVE_UNIX_DOMAIN_SOCKETS

Access to the data in this class is using the following attributes:

data

Keyed as above, each item is a dictionary of lists, corresponding to a column and row lookup from the table data. For example, the first line’s State is [‘State’][0]

Type:

dict

datalist

Keyed as above, each item is a list of dictionaries corresponding to a row and column lookup from the table. For example, the first line’s State is [0][‘State’]

Type:

dict

lines

Keyed as above, each item is a list of the original line of data from the input, in the same order that the data appears in the datalist attribute’s list.

Type:

dict

The keys in the data dictionary and each element of the datalist lists are the same as the headers in the table (e.g. Proto, Recv-Q, etc for ‘Active Internet connections (servers and established)’ and Proto, RefCnt, Flags, etc. for ‘Active UNIX domain sockets (servers and established)’). The datalist row dictionaries also have the following keys:

  • Local IP - (for internet connections) the address portion of the ‘Local Address’ field.

  • Port - (for internet connections) the port portion of the ‘Local Address’ field.

  • PID - the process ID from the ‘PID/Program name’ field.

  • Program name - the process ID from the ‘PID/Program name’ field.

Examples

>>> type(ns)
<class 'insights.parsers.netstat.Netstat'>
>>> sorted(ns.data.keys())  # Both tables stored in dictionary by name
['Active Internet connections (servers and established)', 'Active UNIX domain sockets (servers and established)']
>>> intcons = 'Active Internet connections (servers and established)'
>>> sorted(ns.data[intcons].keys())  # Data stored by column:
['Foreign Address', 'Inode', 'Local Address', 'PID/Program name', 'Proto', 'Recv-Q', 'Send-Q', 'State', 'Timer', 'User']
>>> ns.data[intcons]['Local Address'][1]  # ... and then by row
'127.0.0.1:27017'
>>> ns.datalist[intcons][1]['Local Address']  # Data in a list by row then column
'127.0.0.1:27017'
>>> ns.lines[intcons][1]  # The raw line
'tcp        0      0 127.0.0.1:27017         0.0.0.0:*               LISTEN      184        20380      2007/mongod          off (0.00/0/0)'
>>> ns.get_original_line(intcons, 1)  # Alternative way of getting line
'tcp        0      0 127.0.0.1:27017         0.0.0.0:*               LISTEN      184        20380      2007/mongod          off (0.00/0/0)'
>>> 'qpidd' in ns.running_processes  # All running processes on internet ports
True
>>> 'systemd' in ns.running_processes  # Does not look at UNIX sockets
False
>>> pids = ns.listening_pid  # All PIDs listening on internet ports, with info
>>> sorted(pids.keys())  # Note: keys are strings
['12387', '1272', '1279', '2007']
>>> pids['12387']['addr']
'127.0.0.1'
>>> pids['12387']['port']
'53644'
>>> pids['12387']['name']
'Passenger Rac'
>>> datagrams = ns.search(Type='DGRAM')  # List of data row dictionaries
>>> len(datagrams)
1
>>> datagrams[0]['RefCnt']
'unix  2'
>>> datagrams[0]['Flags']
'[ ]'
>>> datagrams[0]['Type']
'DGRAM'
>>> datagrams[0]['State']
''
>>> datagrams[0]['I-Node']
'11776'
>>> datagrams[0]['PID/Program name']
'1/systemd'
>>> datagrams[0]['Path']
'/run/systemd/shutdownd'
get_original_line(section_id, index)[source]

Get the original netstat line that is stripped white spaces

property listening_pid

Find PIDs of all LISTEN processes

Returns:

If any are found, they are returned in a dictionary following the format:

{'pid': ("addr": ip_address, 'port': port, 'name': process_name)}

Return type:

dict

parse_content(content)[source]

This method must be implemented by classes based on this class.

property running_processes

List all the running processes given in the Active Internet Connections part of the netstat output.

Returns:

set of process names (with spaces, as given in netstat output)

Return type:

set

search(**kwargs)[source]

Search for rows in the data matching keywords in the search.

This method searches both the active internet connections and active UNIX domain sockets. If you only want to search one, specify the name via the search_list keyword, e.g.:

from insights.parsers import Netstat, ACTIVE_UNIX_DOMAIN_SOCKETS
conns.search(search_list=[ACTIVE_UNIX_DOMAIN_SOCKETS], State='LISTEN')

The search_list can be either a list, or a string, containing one of the named constants defined in this module. If search_list is not given, both the active internet connections and active UNIX domain sockets are searched, in that order.

The results of the search are compiled into one list. This allows you to search for all listening processes, whether for internet connections or UNIX sockets, by e.g.:

conns.search(State__contains='LISTEN')

This method uses the insights.parsers.keyword_search() function - see its documentation for a complete description of its keyword recognition capabilities.

class insights.parsers.netstat.NetstatAGN(context, extra_bad_lines=None)[source]

Bases: CommandParser

Parse the netstat -agn command to get interface multicast infomation.

Sample command output:

IPv6/IPv4 Group Memberships
Interface       RefCnt Group
--------------- ------ ---------------------
lo              1      224.0.0.1
eth0            1      224.0.0.1
lo              3      ff02::1
eth0            4      ff02::1
eth0            1      ff01::1

Examples

>>> type(multicast)
<class 'insights.parsers.netstat.NetstatAGN'>
>>> multicast.data[0]['interface']  # Access by row
'lo'
>>> multicast.data[0]['refcnt']  # Values are strings
'1'
>>> multicast.data[0]['group']  # Column names are lower case
'224.0.0.1'
>>> mc_ifs = multicast.group_by_iface()  # Lists by interface name
>>> len(mc_ifs['lo'])
2
>>> mc_ifs['eth0'][1]['refcnt']  # Listed in order of appearance
'4'
group_by_iface()[source]
Group Netstat AGN data by Iface name, return like this:
>>> content= '''
... {
...     "lo":[
...         {"refcnt":"1", "group":"224.0.0.1"},
...         {"refcnt":"1", "group":"ff02::1"}
...     ]
... }
... '''
parse_content(content)[source]

This method must be implemented by classes based on this class.

class insights.parsers.netstat.NetstatS(context, extra_bad_lines=None)[source]

Bases: LegacyItemAccess, CommandParser

Parses data from the netstat -s command.

The output of the netstat -s command looks like:

Ip:
    3405107 total packets received
    0 forwarded
    0 incoming packets discarded
    2900146 incoming packets delivered
    2886201 requests sent out
    456 outgoing packets dropped
    4 fragments received ok
    8 fragments created
Icmp:
    114 ICMP messages received
    0 input ICMP message failed.
    ICMP input histogram:
        destination unreachable: 107
        echo requests: 4
        echo replies: 3
    261 ICMP messages sent
    0 ICMP messages failed
    ICMP output histogram:
        destination unreachable: 254
        echo request: 3
        echo replies: 4
IcmpMsg:
        InType0: 3
        InType3: 107
        InType8: 4
        OutType0: 4
        OutType3: 254
        OutType8: 3
Tcp:
    1648 active connections openings
    1525 passive connection openings
    105 failed connection attempts
    69 connection resets received
    139 connections established
    2886370 segments received
    2890303 segments send out
    428 segments retransmited
    0 bad segments received.
    212 resets sent
Udp:
    4901 packets received
    107 packets to unknown port received.
    0 packet receive errors
    1793 packets sent
    0 receive buffer errors
    0 send buffer errors

Examples

>>> type(stats)
<class 'insights.parsers.netstat.NetstatS'>
>>> sorted(stats.data.keys())  # Stored by heading, lower case
['icmp', 'icmpmsg', 'ip', 'ipext', 'tcp', 'tcpext', 'udp', 'udplite']
>>> 'ip' in stats.data
True
>>> 'forwarded' in stats.data['ip']   # Then by keyword and value
True
>>> stats.data['ip']['forwarded']  # Values are strings
'0'
>>> stats['ip']['forwarded']  # Direct access via LegacyItemAccess
'0'
>>> stats['ip']['requests_sent_out']  # Spaces converted to underscores
'2886201'
>>> stats['tcp']['bad_segments_received']  # Dots are removed
'0'
>>> stats['icmp']['icmp_output_histogram']['destination_unreachable'] # Sub-table
'254'
parse_content(content)[source]

This method must be implemented by classes based on this class.

class insights.parsers.netstat.Netstat_I(context, extra_bad_lines=None)[source]

Bases: CommandParser

Parse the netstat -i command output to get interface traffic info such as “TX-OK” and “RX-OK”.

The output of netstat -i looks like:

Kernel Interface table
Iface       MTU Met    RX-OK RX-ERR RX-DRP RX-OVR    TX-OK TX-ERR TX-DRP TX-OVR Flg
bond0      1500   0   845265      0      0      0     1753      0      0      0 BMmRU
bond1      1500   0   842447      0      0      0     4233      0      0      0 BMmRU
eth0       1500   0   422518      0      0      0     1703      0      0      0 BMsRU
eth1       1500   0   422747      0      0      0       50      0      0      0 BMsRU
eth2       1500   0   421192      0      0      0     3674      0      0      0 BMsRU
eth3       1500   0   421255      0      0      0      559      0      0      0 BMsRU
lo        65536   0        0      0      0      0        0      0      0      0 LRU

Examples

>>> type(traf)
<class 'insights.parsers.netstat.Netstat_I'>
>>> traf.data[0]['Iface']  # A list of the interfaces and stats.
'bond0'
>>> 'bond0' in traf.group_by_iface  # A dictionary keyed on interface.
True
>>> 'enp0s25' in traf.group_by_iface
False
>>> 'MTU' in traf.group_by_iface['bond0']
True
>>> traf.group_by_iface['bond0']['MTU']  # as string
'1500'
>>> traf.group_by_iface['bond0']['RX-OK']
'845265'
parse_content(content)[source]

This method must be implemented by classes based on this class.

class insights.parsers.netstat.ProcNsat(context)[source]

Bases: Parser

Parse the content of the /proc/net/netstat file

Sample input data looks like:

TcpExt: SyncookiesSent SyncookiesRecv SyncookiesFailed EmbryonicRsts PruneCalled RcvPruned OfoPruned OutOfWindowIcmps LockDroppedIcmps ArpFilter TW TWRecycled TWKilled PAWSPassive PAWSActive PAWSEstab DelayedACKs DelayedACKLocked DelayedACKLost ListenOverflows ListenDrops TCPPrequeued TCPDirectCopyFromBacklog TCPDirectCopyFromPrequeue TCPPrequeueDropped TCPHPHits TCPHPHitsToUser TCPPureAcks TCPHPAcks TCPRenoRecovery TCPSackRecovery TCPSACKReneging TCPFACKReorder TCPSACKReorder TCPRenoReorder TCPTSReorder TCPFullUndo TCPPartialUndo TCPDSACKUndo TCPLossUndo TCPLostRetransmit TCPRenoFailures TCPSackFailures TCPLossFailures TCPFastRetrans TCPForwardRetrans TCPSlowStartRetrans TCPTimeouts TCPLossProbes TCPLossProbeRecovery TCPRenoRecoveryFail TCPSackRecoveryFail TCPSchedulerFailed TCPRcvCollapsed TCPDSACKOldSent TCPDSACKOfoSent TCPDSACKRecv TCPDSACKOfoRecv TCPAbortOnData TCPAbortOnClose TCPAbortOnMemory TCPAbortOnTimeout TCPAbortOnLinger TCPAbortFailed TCPMemoryPressures TCPSACKDiscard TCPDSACKIgnoredOld TCPDSACKIgnoredNoUndo TCPSpuriousRTOs TCPMD5NotFound TCPMD5Unexpected TCPSackShifted TCPSackMerged TCPSackShiftFallback TCPBacklogDrop PFMemallocDrop TCPMinTTLDrop TCPDeferAcceptDrop IPReversePathFilter TCPTimeWaitOverflow TCPReqQFullDoCookies TCPReqQFullDrop TCPRetransFail TCPRcvCoalesce TCPOFOQueue TCPOFODrop TCPOFOMerge TCPChallengeACK TCPSYNChallenge TCPFastOpenActive TCPFastOpenActiveFail TCPFastOpenPassive TCPFastOpenPassiveFail TCPFastOpenListenOverflow TCPFastOpenCookieReqd TCPSpuriousRtxHostQueues BusyPollRxPackets TCPAutoCorking TCPFromZeroWindowAdv TCPToZeroWindowAdv TCPWantZeroWindowAdv TCPSynRetrans TCPOrigDataSent TCPHystartTrainDetect TCPHystartTrainCwnd TCPHystartDelayDetect TCPHystartDelayCwnd TCPACKSkippedSynRecv TCPACKSkippedPAWS TCPACKSkippedSeq TCPACKSkippedFinWait2 TCPACKSkippedTimeWait TCPACKSkippedChallenge TCPWqueueTooBig
TcpExt: 10 20 30 40 0 0 0 0 0 0 8387793 2486 0 0 0 3 27599330 35876 309756 0 0 84351589 9652226708 54271044841 0 10507706759 112982361 177521295 3326559442 0 26212 0 36 33090 0 14345 959 8841 425 833 399 0 160 2 633809 11063 7056 233144 1060065 640242 0 228 54 0 310709 0 820887 112 900268 31664 0 232144 0 0 0 261 1048 808390 9 0 0 120433 244126 450077 0 0 0 5625 0 0 0 0 0 6772744900 19251701 0 0 465 463 0 0 0 0 0 0 1172 0 623074473 51282 51282 142025 465090 8484708872 836920 18212118 88 4344 0 0 5 4 3 2 1
IpExt: InNoRoutes InTruncatedPkts InMcastPkts OutMcastPkts InBcastPkts OutBcastPkts InOctets OutOctets InMcastOctets OutMcastOctets InBcastOctets OutBcastOctets InCsumErrors InNoECTPkts InECT1Pkts InECT0Pkts InCEPkts ReasmOverlaps
IpExt: 100 200 300 400 500 0 10468977960762 8092447661930 432 0 3062938 0 0 12512350267 400 300 200 100

Examples

>>> type(pnstat)
<class 'insights.parsers.netstat.ProcNsat'>
>>> len(pnstat.data) == 132
True
>>> pnstat.get_stats('ReasmOverlaps')
100
>>> pnstat.get_stats('EmbryonicRsts')
40
get_stats(key_stats)[source]
(int): The parser method will return the integer stats of the key if key is present in

TcpExt or IpExt else it will return None.

parse_content(content)[source]

This method must be implemented by classes based on this class.

class insights.parsers.netstat.SsTULPN(context, extra_bad_lines=None)[source]

Bases: CommandParser

Parse the output of the /usr/sbin/ss -tulpn command.

This class parse the input as a table with header:

“Netid State Recv-Q Send-Q Local-Address-Port Peer-Address-Port Process”

Sample input data looks like:

Netid  State      Recv-Q Send-Q Local Address:Port               Peer Address:Port
udp    UNCONN     0      0                  *:55898                 *:*
udp    UNCONN     0      0          127.0.0.1:904                   *:*                   users:(("rpc.statd",pid=29559,fd=7))
udp    UNCONN     0      0                  *:111                   *:*                   users:(("rpcbind",pid=953,fd=9))
udp    UNCONN     0      0                 :::37968                :::12345               users:(("rpc.statd",pid=29559,fd=10))
tcp    LISTEN     0      128                *:111                   *:*                   users:(("rpcbind",pid=1139,fd=5),("systemd",pid=1,fd=41))

Examples

>>> type(ss)
<class 'insights.parsers.netstat.SsTULPN'>
>>> sorted(ss.data[1].keys())  # Rows stored by column headings
['Local-Address-Port', 'Netid', 'Peer-Address-Port', 'Process', 'Recv-Q', 'Send-Q', 'State']
>>> ss.data[0]['Local-Address-Port']
'*:55898'
>>> ss.data[0]['State']
'UNCONN'
>>> rpcbind = ss.get_service("rpcbind")  # All connections opened by rpcbind
>>> len(rpcbind)
2
>>> rpcbind[0]['State']
'UNCONN'
>>> rpcbind[1]['State']
'LISTEN'
>>> rpcbind[0]['Process']
'users:(("rpcbind",pid=953,fd=9))'
>>> rpcbind[1]['Process']
'users:(("rpcbind",pid=1139,fd=5),("systemd",pid=1,fd=41))'
>>> using_55898 = ss.get_port("55898")  # Both local and peer port searched
>>> len(using_55898)
1
>>> 'Process' in using_55898  # Not in dictionary if field not found
False
>>> rpcbind == ss.get_localport('111')  # Only local port or address searched
True
parse_content(content)[source]

This method must be implemented by classes based on this class.

class insights.parsers.netstat.SsTUPNA(context, extra_bad_lines=None)[source]

Bases: SsTULPN

Parse the output of the /usr/sbin/ss -tupna command.

This class parse the input as a table with header:

“Netid State Recv-Q Send-Q Local-Address-Port Peer-Address-Port Process”

Sample input data looks like:

Netid State      Recv-Q Send-Q    Local Address:Port    Peer Address:Port
tcp   UNCONN     0      0                     *:68                 *:*      users:(("dhclient",1171,6))
tcp   LISTEN     0      100           127.0.0.1:25                 *:*      users:(("master",1326,13))
tcp   ESTAB      0      0         192.168.0.106:22     192.168.0.101:59232  users:(("sshd",11427,3))
tcp   ESTAB      0      0         192.168.0.106:739    192.168.0.105:2049
tcp   LISTEN     0      128                  :::111               :::*      users:(("rpcbind",483,11))

Examples

>>> type(ssa)
<class 'insights.parsers.netstat.SsTUPNA'>
>>> sorted(ssa.data[2].items())
[('Local-Address-Port', '192.168.0.106:22'), ('Netid', 'tcp'), ('Peer-Address-Port', '192.168.0.101:59232'), ('Process', 'users:(("sshd",11427,3))'), ('Recv-Q', '0'), ('Send-Q', '0'), ('State', 'ESTAB')]
>>> sorted(ssa.get_service("sshd")[0].items())  # All connections opened by rpcbind
[('Local-Address-Port', '192.168.0.106:22'), ('Netid', 'tcp'), ('Peer-Address-Port', '192.168.0.101:59232'), ('Process', 'users:(("sshd",11427,3))'), ('Recv-Q', '0'), ('Send-Q', '0'), ('State', 'ESTAB')]
>>> sorted(ssa.get_port("2049")[0].items())  # Both local and peer port searched
[('Local-Address-Port', '192.168.0.106:739'), ('Netid', 'tcp'), ('Peer-Address-Port', '192.168.0.105:2049'), ('Recv-Q', '0'), ('Send-Q', '0'), ('State', 'ESTAB')]
>>> sorted(ssa.get_localport("739")[0].items())  # local port searched
[('Local-Address-Port', '192.168.0.106:739'), ('Netid', 'tcp'), ('Peer-Address-Port', '192.168.0.105:2049'), ('Recv-Q', '0'), ('Send-Q', '0'), ('State', 'ESTAB')]
>>> sorted(ssa.get_peerport("59232")[0].items())  # peer port searched
[('Local-Address-Port', '192.168.0.106:22'), ('Netid', 'tcp'), ('Peer-Address-Port', '192.168.0.101:59232'), ('Process', 'users:(("sshd",11427,3))'), ('Recv-Q', '0'), ('Send-Q', '0'), ('State', 'ESTAB')]
parse_content(content)[source]

This method must be implemented by classes based on this class.