Custom Datasources Catalog

insights.specs.datasources

Custom datasources provide functionality beyond simple_file and simple_command type datasources. If the custom datasource is short and concise it may be added to this module. Other datasources should be added as a separate module. Normally custom datasources are necessary for core-collection. In order for a custom datasource to execute, all of its dependencies must be explicitly loaded by the client. The client uses the YAML template insights.collect.default_manifest and each parser/combiner/component required by a custom datasource must be included in the YAML template to ensure it is loaded.

insights.specs.datasources.DEFAULT_SHELL_TIMEOUT = 10

Default timeout in seconds for ctx.shell_out() commands, must be provided as an arg

Type

int

insights.specs.datasources.get_running_commands(ps, ctx, commands)

Search for a list of commands in Ps combiner output and returns the full path to the command

Parameters

• ps -- Ps combiner object

• ctx -- Context of the current collection

• commands (list) -- List of commands to search for in ps output

Returns

List of the full command paths of the all command.

Return type

list

Raises

TypeError -- Raised when commands args is not a list

insights.specs.datasources.aws

Custom datasources for aws information

class insights.specs.datasources.aws.LocalSpecs

Bases: Specs

Local specs used only by aws datasources

aws_imdsv2_token = <insights.core.spec_factory.simple_command object>

context_handlers = {}

registry = {}

insights.specs.datasources.aws.aws_imdsv2_token(broker)

This datasource provides a session token for use by other specs to collect metadata information on AWS EC2 nodes with IMDSv2 support..

Typical output of the input spec, which is also the output of this datasource:

AQAEABcCFaLcKRfXhLV9_ezugiVzra-qMBoPbdWGLrbdfqSLEJzP8w==

Returns

String that is the actual session token to be used in other commands

Return type

str

Raises

SkipComponent -- When an error occurs or no token is generated

insights.specs.datasources.awx_manage

Custom datasources for awx_manage information

class insights.specs.datasources.awx_manage.LocalSpecs

Bases: Specs

Local specs used only by awx_manage datasources

awx_manage_check_license_data_raw = <insights.core.spec_factory.simple_command object>

Returns the output of command /usr/bin/awx-manage check_license --data

context_handlers = {}

registry = {}

insights.specs.datasources.awx_manage.awx_manage_check_license_data_datasource(broker)

This datasource provides the not-sensitive information collected from /usr/bin/awx-manage check_license --data.

Typical content of /usr/bin/awx-manage check_license --data file is:

{"contact_email": "test@redhat.com", "company_name": "test Inc", "instance_count": 100, "license_date": 1655092799, "license_type": "enterprise", "subscription_name": "Red Hat Ansible Automation, Standard (100 Managed Nodes)", "sku": "MCT3691", "support_level": "Standard", "product_name": "Red Hat Ansible Automation Platform", "valid_key": true, "satellite": null, "pool_id": "2c92808179803e530179ea5989a157a4", "current_instances": 1, "available_instances": 100, "free_instances": 99, "time_remaining": 29885220, "trial": false, "grace_period_remaining": 32477220, "compliant": true, "date_warning": false, "date_expired": false}

Returns

JSON string containing non-sensitive information.

Return type

str

Raises

SkipComponent -- When the filter/path does not exist or any exception occurs.

insights.specs.datasources.candlepin_broker

Custom datasources for candlepin broker.xml

class insights.specs.datasources.candlepin_broker.LocalSpecs

Bases: Specs

Local specs used only by candlepin_broker datasources

candlepin_broker_input = <insights.core.spec_factory.simple_file object>

Returns the contents of the file /etc/candlepin/broker.xml

context_handlers = {}

registry = {}

insights.specs.datasources.candlepin_broker.candlepin_broker(broker)

This datasource provides the candlepn broker configuration information collected from /etc/candlepin/broker.xml.

Typical content of /etc/candlepin/broker.xml file is:

<configuration xmlns="urn:activemq"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="urn:activemq /schema/artemis-configuration.xsd">
    <core xmlns="urn:activemq:core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="urn:activemq:core ">
        <acceptors>
            <acceptor name="in-vm">vm://0</acceptor>
            <acceptor name="stomp">tcp://localhost:61613?protocols=STOMP;useEpoll=false;sslEnabled=true;trustStorePath=/etc/candlepin/certs/truststore;trustStorePassword=CDX9i3K5uPPBzcNtzz5tcycVf5PuXA5w;keyStorePath=/etc/candlepin/certs/keystore;keyStorePassword=4iBpTS45VZjFmVdNzRhRKNXtxbsH5Dij;needClientAuth=true</acceptor>
        </acceptors>
        <security-enabled>true</security-enabled>
    </core>
</configuration>

Note

This datasource may be executed using the following command:

insights cat --no-header candlepin_broker

Returns

XML string after removeing sensitive information.

Return type

str

Raises

SkipComponent -- When the path does not exist or any exception occurs.

insights.specs.datasources.cloud_init

Custom datasources for cloud initialization information

class insights.specs.datasources.cloud_init.LocalSpecs

Bases: Specs

Local specs used only by cloud_init datasources

cloud_cfg_input = <insights.core.spec_factory.simple_file object>

Returns the contents of the file /etc/cloud/cloud.cfg

context_handlers = {}

registry = {}

insights.specs.datasources.cloud_init.cloud_cfg(broker)

This datasource provides configuration of /etc/cloud/cloud.cfg file.

Note

Since this file may contain sensitive information, it should be filtered before the Insights collecting it. The filters will be added via the insights.specs.Specs.cloud_cfg Spec. If nothing is added to the filter, nothing will be collected.

Typical content of /etc/cloud/cloud.cfg file is:

#cloud-config
users:
  - name: demo
    ssh-authorized-keys:
      - key_one
      - key_two
    passwd: $6$j212wezy$7H/1LT4f9/N3wpgNunhsIqtMj62OKiS3nyNwuizouQc3u7

ssh_deletekeys: 1

network:
    version: 1
    config:
      - type: physical
        name: eth0
        subnets:
          - type: dhcp
          - type: dhcp6

system_info:
    default_user:
    name: user2
    plain_text_passwd: 'someP@assword'
    home: /home/user2

debug:
    output: /var/log/cloud-init-debug.log
    verbose: true

Returns

YAML string after removing the sensitive information.

Return type

str

Raises

SkipComponent -- When the path does not exist, nothing is collected, or any exception occurs.

insights.specs.datasources.corosync

Custom datasources related to corosync

insights.specs.datasources.corosync.corosync_cmapctl_cmds(broker)

corosync-cmapctl use different arguments on RHEL7 and RHEL8.

Returns

A list of related corosync-cmapctl commands based the RHEL version.

Return type

list

insights.specs.datasources.dir_list

Custom datasources to get a list of directories to check disk size.

insights.specs.datasources.dir_list.du_dir_list(broker)

Return a list of directories from the spec filter

insights.specs.datasources.ethernet

Custom datasource for gathering a list of the ethernet interface names.

class insights.specs.datasources.ethernet.LocalSpecs

Bases: Specs

Local specs used only by ethernet_interfaces datasource.

context_handlers = {}

ip_link = <insights.core.spec_factory.simple_command object>

registry = {}

insights.specs.datasources.ethernet.interfaces(broker)

This datasource provides a list of the ethernet interfaces available.

Typical content of the spec is:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000\    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000\    link/ether 52:54:00:13:14:b5 brd ff:ff:ff:ff:ff:ff
3: enp8s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000\    link/ether 52:54:00:e5:11:d4 brd ff:ff:ff:ff:ff:ff
4: enp1s0.2@enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000\    link/ether 52:54:00:13:14:b5 brd ff:ff:ff:ff:ff:ff
5: ib0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 4092 qdisc mq state DOWN group default qlen 256\    link/infiniband 00:01:02:03:fd:90:0:00:00:00:00:00:ef:0d:8b:02:01:d9:82:fd

Note

This datasource may be executed using the following command:

insights cat --no-header ethernet_interfaces

Sample data returned:

['enp1s0', 'enp8s0', 'enp1s0.2']

Returns

List of the ethernet interfaces available.

Return type

list

Raises

SkipComponent -- When there is not any content.

insights.specs.datasources.httpd

Custom datasources related to httpd

insights.specs.datasources.httpd.httpd_cmds(broker)

Function to search the output of ps auxcww to find all running Apache webserver processes and extract the binary path.

Returns

List of the binary paths to each running process

Return type

list

insights.specs.datasources.httpd.httpd_on_nfs(broker)

Function to get the count of httpd opened file on nfs v4

Returns

JSON string with keys: “httpd_ids”, “nfs_mounts”, “open_nfs_files”

Return type

str

insights.specs.datasources.ipcs

Custom datasources to get the semid of all the inter-processes.

insights.specs.datasources.ipcs.semid(broker)

This datasource provides a list of the semid of all the inter-processes.

Note

This datasource may be executed using the following command:

insights cat --no-header ipcs_s_i

Sample output:

[
    '65570', '98353', '98354'
]

Returns

A list of the semid of all the inter-processes.

Return type

list

insights.specs.datasources.kernel_module_list

Custom datasources to get a list of modules to check the detailed module info.

insights.specs.datasources.kernel_module_list.kernel_module_filters(broker)

Return a string of a list of modules from the spec filter, separated with space.

insights.specs.datasources.lpstat

Custom datasources for lpstat information

class insights.specs.datasources.lpstat.LocalSpecs

Bases: Specs

Local specs used only by lpstat datasources

context_handlers = {}

lpstat_v = <insights.core.spec_factory.simple_command object>

Returns the output of command /usr/bin/lpstat -v

registry = {}

insights.specs.datasources.lpstat.lpstat_protocol_printers_info(broker)

This datasource provides the not-sensitive information collected from /usr/bin/lpstat -v.

Typical content of /usr/bin/lpstat -v file is:

"device for test_printer1: ipp://cups.test.com/printers/test_printer1"

Returns

Returns the collected content containing non-sensitive information

Return type

DatasourceProvider

Raises

SkipComponent -- When the filter/path does not exist or any exception occurs.

insights.specs.datasources.luks_devices

Custom datasource for gathering a list of encrypted LUKS block devices and their properties.

class insights.specs.datasources.luks_devices.LocalSpecs

Bases: Specs

Local specs used only by LUKS_data_sources datasource.

context_handlers = {}

cryptsetup_luks_dump_commands = <insights.core.spec_factory.foreach_execute object>

cryptsetup_luks_dump_token_commands = <insights.core.spec_factory.foreach_execute object>

registry = {}

insights.specs.datasources.luks_devices.luks_block_devices(broker)

This datasource provides a list of LUKS encrypted device.

Sample data returned:

['/dev/sda', '/dev/nvme0n1p3']

Returns

List of the LUKS encrypted block devices.

Return type

list

Raises

• SkipComponent -- When there is not any LUKS encrypted block device on the

• system. --

insights.specs.datasources.luks_devices.luks_data_sources(broker)

This datasource provides the output of ‘cryptsetup luksDump’ command for every LUKS encrypted device on the system. The digest and salt fields are filtered out as they can be potentially sensitive.

Returns

List of outputs of the cryptsetup luksDump command.

Return type

list

Raises

• SkipComponent -- When there is not any LUKS encrypted block device on the

• system. --

insights.specs.datasources.malware_detection

insights.specs.datasources.md5chk

Custom datasources related to md5

insights.specs.datasources.md5chk.files(broker)

Return a list of files to be processed by the md5chk_files spec

insights.specs.datasources.package_provides

Custom datasource for package_provides

insights.specs.datasources.package_provides.cmd_and_pkg(broker)

Collect a list of running commands and the associated RPM package providing those commands. The commands are based on filters so rules must add the desired commands as filters to enable collection. If a command is not provided by an RPM then it will not be included in the output.

In order for the full command line to be present in the Ps combiner a filter must be added to the spec ps_auxww. A filter must also be added to package_provides_command so this datasource will look for the command in Ps.

Parameters

broker -- the broker object for the current session

Returns

Returns the collected information as a file with 1 line per command

Return type

DatasourceProvider

Raises

SkipComponent -- Raised if no data is collected

insights.specs.datasources.package_provides.get_package(ctx, file_path)

Get the RPM package that owns the specified filename with path

Parameters

• ctx -- The current execution context

• file_path (str) -- The full path and filename for RPM query

Returns

The name of the RPM package that provides the file or None if file is not associated with an RPM.

Return type

str

insights.specs.datasources.pcp

Custom datasource related PCP (Performance Co-Pilot)

insights.specs.datasources.pcp.pcp_enabled(broker)

Returns

True if pmproxy service is on in services

Return type

bool

Raises

SkipComponent -- When pmproxy service is not enabled

insights.specs.datasources.pcp.pmlog_summary_args(broker)

Determines the pmlogger file and the metrics to collect via pmlog_summary spec.

Returns

Full arguments string that will be passed to the pmlogsummary,

which contains the pmlogger archive file and the required metrics.

Return type

str

Raises

SkipComponent -- Raises when meeting one of the following scenario: - No pmlogger process is running - No pmlogger file - No “mandatory on” metrics in config.ros

insights.specs.datasources.ps

Custom datasources for ps information

class insights.specs.datasources.ps.LocalSpecs

Bases: Specs

Local specs used only by ps datasources

context_handlers = {}

ps_eo_args = <insights.core.spec_factory.simple_command object>

Returns ps output including pid and full args

registry = {}

insights.specs.datasources.ps.ps_eo_cmd(broker)

Custom datasource to collect the full paths to all running commands on the system provided by the ps -eo pid,args command. After collecting the data, all of the args are trimmed to leave only the command including full path.

Sample output from the ps -eo pid, args command:

PID COMMAND
  1 /usr/lib/systemd/systemd --switched-root --system --deserialize 31
  2 [kthreadd]
  3 [rcu_gp]
  4 [rcu_par_gp]
  6 [kworker/0:0H-events_highpri]
  9 [mm_percpu_wq]
 10 [rcu_tasks_kthre]
 11 /usr/bin/python3 /home/user1/python_app.py
 12 [kworker/u16:0-kcryptd/253:0]

This datasource trims off the args to minimize possible PII and sensitive information. After trimming the data looks like this:

PID COMMAND
  1 /usr/lib/systemd/systemd
  2 [kthreadd]
  3 [rcu_gp]
  4 [rcu_par_gp]
  6 [kworker/0:0H-events_highpri]
  9 [mm_percpu_wq]
 10 [rcu_tasks_kthre]
 11 /usr/bin/python3
 12 [kworker/u16:0-kcryptd/253:0]

Returns

Returns a multiline string in the same format as ps output

Return type

str

Raises

SkipComponent -- Raised if no data is available

insights.specs.datasources.sap

Custom datasources for SAP related specs

class insights.specs.datasources.sap.LocalSpecs

Bases: Specs

Local specs used only by sap datasources

context_handlers = {}

registry = {}

sap_hana_instance()

list: List of the SAP HANA Instances.

sap_instance()

list: List of all SAP Instances.

insights.specs.datasources.sap.ld_library_path_of_user(broker)

list: The list of “Username LD_LIBRARY_PATH”, e.g.:

[
  'sr1adm /usr/sap/RH1/SYS/exe/run:/usr/lib/',
  'sr2adm /usr/sap/RH2/SYS/exe/run',
]

Note

Currently, only Sap users are supported.

insights.specs.datasources.sap.sap_hana_sid(broker)

list: List of the SID of SAP HANA Instances.

insights.specs.datasources.sap.sap_hana_sid_SID_nr(broker)

list: List of tuples (sid, SID, Nr) of SAP HANA Instances.

insights.specs.datasources.sap.sap_sid(broker)

list: List of the SID of all the SAP Instances.

insights.specs.datasources.satellite_missed_queues

class insights.specs.datasources.satellite_missed_queues.LocalSpecs

Bases: Specs

Local specs used only by get_satellite_missed_pulp_agent_queues datasources

content_host_uuids = <insights.core.spec_factory.simple_command object>

context_handlers = {}

qpid_queues = <insights.core.spec_factory.simple_command object>

registry = {}

insights.specs.datasources.satellite_missed_queues.satellite_missed_pulp_agent_queues(broker)

This datasource provides the missed pulp agent queues information on satellite server.

Note

This datasource may be executed using the following command:

insights cat --no-header satellite_missed_pulp_agent_queues

Sample output:

pulp.agent.09008eec-aba6-4174-aa9f-e930004ce5c9:2018-01-16 00:06:13
pulp.agent.fac7ebbc-ee4f-44b4-9fe0-3f4e42c7f024:2018-01-16 00:06:16
0

Returns

All the missed pulp agent queues and the boolean mark if the data is

truncated in the last line. If the value of last line is 0, it means all the missed queues are returned. If the value of the last line is 1, it means there are a lot of missed queues, to avoid render error, only the first 10 missed queues are returned.

Return type

str

Raises

SkipComponent -- When the error doen’t happen or the missed queues have been recreated.

insights.specs.datasources.ssl_certificate

Custom datasource to get ssl certificate file path.

insights.specs.datasources.ssl_certificate.httpd_certificate_info_in_nss(broker)

Get the certificate info configured in nss database

Parameters

broker -- the broker object for the current session

Returns

Returns a list of tuple with the Nss database path and the certificate nickname

Return type

list

Raises

SkipComponent -- Raised when NSSEngine isn’t enabled or “NSSCertificateDatabase” and “NSSNickname” directives aren’t found

insights.specs.datasources.ssl_certificate.httpd_ssl_certificate_files(broker)

Get the httpd SSL certificate file path configured by “SSLCertificateFile”

Parameters

broker -- the broker object for the current session

Returns

Returns the SSL certificate file path configured by “SSLCertificateFile”

Return type

str

Raises

SkipComponent -- Raised if “SSLCertificateFile” directive isn’t found

insights.specs.datasources.ssl_certificate.mssql_tls_cert_file(broker)

Get the mssql tls certificate file path configured by “ssl_certificate” :param broker: the broker object for the current session

Returns

Returns the SSL certificate file path configured by “ssl_certificate”

Return type

str

Raises

SkipComponent -- Raised if “ssl_certificate” directive isn’t found

insights.specs.datasources.ssl_certificate.nginx_ssl_certificate_files(broker)

Get the nginx SSL certificate file path configured by “ssl_certificate”

Parameters

broker -- the broker object for the current session

Returns

Returns the SSL certificate file path configured by “ssl_certificate”

Return type

str

Raises

SkipComponent -- Raised if “ssl_certificate” directive isn’t found

insights.specs.datasources.sys_fs_cgroup_memory_tasks_number

Custom datasources for the number of “tasks” files under /sys/fs/cgroup/memory

class insights.specs.datasources.sys_fs_cgroup_memory_tasks_number.LocalSpecs

Bases: Specs

Local specs used only by sys_fs_cgroup_memory_tasks_number datasources

context_handlers = {}

registry = {}

sys_fs_cgroup_memory_tasks_raw = <insights.core.spec_factory.simple_command object>

Returns the output of command /usr/bin/find /sys/fs/cgroup/memory -name 'tasks'

insights.specs.datasources.sys_fs_cgroup_memory_tasks_number.sys_fs_cgroup_memory_tasks_number_data_datasource(broker)

This datasource provides the numeber of “tasks” file collected from /usr/bin/find /sys/fs/cgroup/memory -name 'tasks'.

Typical content of /usr/bin/find /sys/fs/cgroup/memory -name 'tasks' command is:

/sys/fs/cgroup/memory/user.slice/tasks
/sys/fs/cgroup/memory/system.slice/rh-nginx120-nginx.service/tasks
/sys/fs/cgroup/memory/system.slice/named.service/tasks
/sys/fs/cgroup/memory/system.slice/rhel-push-plugin.service/tasks

Returns

the number of “tasks” file under /sys/fs/cgroup/memory

Return type

string

Raises

SkipComponent -- When any exception occurs.

insights.specs.datasources.system_user_dirs

Custom datasource for CVE-2021-35937, CVE-2021-35938, and CVE-2021-35939.

insights.specs.datasources.system_user_dirs.system_user_dirs(broker)

Custom datasource for CVE-2021-35937, CVE-2021-35938, and CVE-2021-35939.

It collects package names from the rpm -qa --qf="[%{=NAME}; %{FILEMODES:perms}; %{FILEUSERNAME}; %{FILEGROUPNAME}\n]" command, if the package has at least one directory which is writable by a specific user/group or the others.

Raises

SkipComponent -- Raised if no data is available

Returns

Sorted list of package names

Return type

List[str]

insights.specs.datasources.user_group

Custom datasources for user and group related specs

Note

It should be noted that the specs in this module might contains sensitive information, please avoid collecting any of them.

insights.specs.datasources.user_group.group_filters(broker)

Return a string contains the list of groups getting from the spec filter, separated with space.

insights.specs.datasources.yum_updates

Custom datasource for collecting yum updates

insights.specs.datasources.yum_updates.yum_updates(_broker)

This datasource provides a list of available updates on the system. It uses the yum python library installed locally, and collects list of available package updates, along with advisory info where applicable.

Sample data returned:

{
  "releasever": "8",
  "basearch": "x86_64",
  "update_list": {
    "NetworkManager-1:1.22.8-4.el8.x86_64": {
      "available_updates": [
        {
          "package": "NetworkManager-1:1.22.8-5.el8_2.x86_64",
          "repository": "rhel-8-for-x86_64-baseos-rpms",
          "basearch": "x86_64",
          "releasever": "8",
          "erratum": "RHSA-2020:3011"
        }
      ]
    }
  },
  "metadata_time": "2021-01-01T09:39:45Z"
}

Returns

List of available updates

Return type

list

Raises

SkipComponent -- Raised when neither dnf nor yum is found