Insights Core Datasource Registry¶
This notebook shows the recommended method of writing datasources.
It also shows how to register datasources with Insights Core to provide alternative methods of collection while still taking advantage of our parser and combiner catalog.
It assumes familiarity with datasources from the Standard Components section of the Insights Core Tutorial.
import sys sys.path.insert(0, "../..")
from insights import run from insights.core import dr from insights.core.spec_factory import simple_file, simple_command
Fixing datasource names¶
The simplest way to define a datasource is with a helper class from
However, if you use one of these to define a datasource at the module level, you’ll notice that it doesn’t have a very useful name.
hosts = simple_file("/etc/hosts") print(dr.get_name(hosts))
We can fix that by including it in a subclass of
This is the recommended way of writing datasources.
from insights.core.spec_factory import SpecSet class MySpecs(SpecSet): hosts = simple_file("/etc/hosts") print(dr.get_name(MySpecs.hosts))
Making datasources dynamic¶
What if you have datasources on which many downstream components depend, and you want to provide different ways of collecting the data they represent? Maybe you want to execute a command in one context but read from a file in another. Parsers depend on a single datasource, and jamming multiple collection methods into a single implementation isn’t attractive.
Instead, you can define a subclass of
insights.core.spec_factory.SpecSet that has
insights.core.spec_factory.RegistryPoint instances instead of regular datasources. Then you can provide implementations for the registry points in the form of datasources that are members of subclasses of your original class. This keeps the alternative implementations cleanly separated while allowing parsers to depend on a single component.
Note that this doesn’t work like normal class inheritance, although it uses the class inheritance mechanism.
from insights.core.spec_factory import RegistryPoint from insights.core.context import ExecutionContext, HostContext # We'll use HostContext and OtherContext as our alternatives. class OtherContext(ExecutionContext): pass
# Define the components that your downstream components should depend on. class TheSpecs(SpecSet): hostname = RegistryPoint() fstab = simple_file("/etc/fstab", context=HostContext) # Provide different implementations for hostname by subclassing TheSpecs and # giving the datasources names that match their corresponding registry points. class HostSpecs(TheSpecs): hostname = simple_command("/usr/bin/hostname", context=HostContext) class OtherSpecs(TheSpecs): hostname = simple_file("/etc/hostname", context=OtherContext) # Note that we don't and actually can't provide an alternative for TheSpecs.fstab # since it's not a RegistryPoint.
Downstream components should depend on
TheSpecs.hostname, and the implementation that actually runs and backs that component will depend on the context in which you run.
results = run(TheSpecs.hostname, context=HostContext) print(results[TheSpecs.hostname])
results = run(TheSpecs.hostname, context=OtherContext) print(results[TheSpecs.hostname])
results = run(TheSpecs.fstab, context=HostContext) print(results[TheSpecs.fstab])
RegistryPoint instances in
SpecSet subclasses are converted to special datasources that simply check their dependencies and return the last one that succeeds. So,
TheSpecs.hostname is just a datasource. When
TheSpecs, the class machinery recognizes that
HostSpecs.hostname is a datasource and is named the same as a
RegistryPoint in an immediate super class. When that happens, the datasource of the subclass is added as a dependency of the
datasource in the superclass.
If the datasources in each subclass depend on different contexts, only one of them will fire. That’s why when we ran with
HostContext, the command was run, but when we ran with
OtherContext, the file was collected.
Notice that the
TheSpecs.fstab datasource can be run, too. If a subclass had provided a datasource of the same name, it would not have been registered with the super class but would instead have stayed local to that subclass.
Note also that the datasources in the alternative implementation classes aren’t special in any other way. You can run them directly, and components can depend on them if you want, although if you’re providing them as an implementation to a registry point, components really should depend on that instead of a particular implementation.
What happens if you have multiple subclass implementations for a given registry point, and more than one of them depends on the same context? In that case, the last one to be registered for that context is the one that runs.
Registering implementations for standard datasources¶
Providing alternative implementations for the standard Insights Core datasources is easy. The datasources on which the core parsers depend are all defined as
RegistryPoints on the
Specs class in
from insights.specs import Specs class UseThisInstead(Specs): hostname = simple_file("/etc/hostname", context=OtherContext) results = run(Specs.hostname, context=OtherContext) print(results[Specs.hostname]) print(results.get(Specs.hosts))
Specs.hosts didn’t run! That’s because we haven’t loaded the module containing the default implementations, and we’ve only provided an implementation for
Specs.hostname. Also, none of the defaults depend on
What if you want to use the default datasources but only want to override a few of them, even for the same context?
Create a subclass that does exactly that:
from pprint import pprint from insights.specs import default # load the default implementations # Note that the default context is HostContext unless otherwise specified # with a context= keyword argument. class SpecialSpecs(Specs): hostname = simple_file("/etc/hostname") results = run(Specs.hostname) # show that the default didn't run pprint(results[Specs.hostname]) pprint(results[SpecialSpecs.hostname]) pprint(results.get(default.DefaultSpecs.hostname, None)) print results = run(Specs.hosts) #show that the default ran pprint(results[Specs.hosts]) pprint(results[default.DefaultSpecs.hosts])
TextFileProvider("'/etc/hostname'") TextFileProvider("'/etc/hostname'") None TextFileProvider("'/etc/hosts'") TextFileProvider("'/etc/hosts'")
If multiple datasources provide implementations for the same registry point and depend on the same context, then the last implementation to load is the one that is executed under that context.