OpenCPI Application Specification (OAS) XML Documents
This section defines the XML document format for describing OpenCPI applications. Such XML documents may be held in files or in text strings within a program. They describe a collection of component instances, along with their interconnections and configuration properties. An OAS may be directly executed using the ocpirun utility program described below. An OAS may also be constructed and/or manipulated programmatically and dynamically, and executed using an API described in the later section: API for Executing XML-based Applications in C++/Python: ACI
The primary contents of the OAS are component instances. When the author of an OAS specifies a component instance, they are referring to a component specification. They are saying: I need a component implementation that meets this specification. Normally, the OAS says only that, and does not say which particular implementation of that component spec (i.e. which worker) should be used. This allows the OAS to be used in a variety of different configurations of hardware and different libraries of component implementations.
A simple example of an OAS is below, showing an application that reads data from a file, adds 1 to each data value, and writes the result.
<application>
<instance component='file_read' connect='add1'>
<property name='filename' value='test.input'/>
</instance>
<instance component='add1' connect='file_write'/>
<instance component='file_write'>
<property name='filename' value='test.outputwrong'/>
</instance>
</application>
Each instance specifies the component, some properties, and a connection.
When used in a development environment and when the application is in a project, this XML file has other attributes unrelated to the actual execution of an application. See the Application XML Files In Projects topic for more details.
Quick XML Introduction
XML documents are text files that contain information formatted according to XML (EXtensible
Markup Language) syntax and structured according to a particular application-specific
schema. The textual XML information itself is formatted into elements, attributes, and
textual content. The OAS XML schema does not use or allow textual content at this time. XML
elements have attributes and child elements (forming a hierarchy of elements). XML
elements take two forms. The simpler one is when an element has no child (embedded) elements and
no textual content. It looks like this (for element of type xyz, with attribute abc):
<xyz abc='123'/>
Thus the element begins with the < character and the element type, and is terminated with the
/> characters. Attributes have values in single or double-quotes. Any white space, indentation,
or new lines can be inserted for readability between the element name and attributes or between
attributes. Thus the above example could also be:
<xyz
abc="123"
/>
When the element has child elements (in this case a child element of type ccc with attribute
cat), it looks like:
<xyz abc="123">
<ccc cat="345"/>
</xyz>
In this case the start of the xyz element (and its attributes), is surrounded by <>, and the
end of the xyz element is indicated by </xyz>. An XML schema defines which elements,
attributes, and child elements the document may contain. Every XML document has a single top-level
element that must be structured (attributes and sub- elements) according to the schema. A
<extension> child element is always legal and always ignored. It can be used when tools want
private embedded elements that are unknown to OpenCPI.
An element can be entered directly (as above) or by referring to a separate file that contains that
element. So the example above might have a file ccc1.xml containing:
<ccc cat="345"/>
And then a top-level file called “xyz1.xml” containing:
<xyz abc="123">
<include href="ccc1.xml"/>
</xyz>
However, the schema specifies which elements are allowed to be top-level elements in any file. All element and attribute names used in OpenCPI are case insensitive.
All attributes are defined with specific data types and formats. When an attribute is defined as the boolean type, the default value (used when the attribute is not specified) is “false” unless otherwise noted. In OpenCPI, element and attribute names are case insensitive.
Top Level Element in an OAS: application
The top-level element in every OAS document (file or string) is the application element.
Application elements contain child elements that are either instance or connection, and have
attributes that are name, finished, package, and maxprocessors. These are described
below.
Name attribute (optional)
The name attribute of an application is simply used in various error messages and other debug
log messages. It has no functional purpose, only documentation and labeling.
Finished attribute (optional)
The finished attribute identifies the instance within the OAS that is used to determine when the
application is “finished” executing. When the indicated instance is “finished”, then the whole
application is considered “finished”. If this attribute is not supplied, the application is
considered “finished” when all its instances are “finished”. The value of this attribute must
match the name attribute of one of the instance elements, described below.
For some applications, and some components, there is no definition or functionality of being “finished”. In this case whatever mechanism started the application must decide when to stop it and shut it down.
Package attribute (optional)
The package attribute of an application is used as a default package prefix for all instances in
the assembly. Any instance’s component attribute that does not have a package prefix (has no
periods) is assumed to be in the package indicated by this attribute. When not specified, the
default package prefix for all components mentioned in the assembly is the package ID of the project
the application is in. If the application is not in an OpenCPI project, the default prefix is:
local.
The prefix of the core OpenCPI component library is ocpi.core. If you are using mostly
components in that library, you might include package='ocpi.core' as an attribute. If you are
using only components specified in your own library of components, you could ignore this attribute
and use no prefixes at all. See the component attribute
of the instance element below.
If the value of this attribute starts with a period, then the project’s package ID is used as a
prefix (or local, if the application is not in a project). Thus if the project’s package ID is
myorg.myproj, and this package attribute is .myutilslib, then the default prefix for all
component attributes is myorg.myproj.myutilslib.
MaxProcessors attribute (optional)
The MaxProcessors attribute indicates the maximum number of processors (containers) that should
be used to run the application. When instances are allocated to processors, an algorithm decides
which processor runs each instance. If this attribute is not set, the default behavior is to spread
the instances across available processors, and use a “round-robin” assignment policy when there are
more instances than processors.
If this numeric parameter is set, it limits the number of processors used, if possible. If more are necessary to host the necessary workers, more will indeed be used in any case. An example of when this attribute is not effective is when the availability of implementations of each instance dictate that more processors are needed, such as when the only implementation available for an instance is for a particular processor, which must then be used.
Instance Elements within the Application Element
The instance element is used as a child of the application element to specify a component
instance in the application. It may have property or slave child elements, and may have
component, name, connect, selection, from, to, slave and external
attributes. For example:
<instance component='file_read' connect='add1'>
<property name='filename' value='test.input'/>
</instance>
The order of instance elements is significant for the purpose of assigning names to each instance when the instance element does not specify a name. When applications are started, the instances in the application are started using ordering rules that do not depend on the ordering of instance elements in the OAS. The detailed ordering rules are outside the scope of this document, but in general instances with no input ports (typically sources of data) are started last to avoid startup overrun conditions.
Component attribute (required)
The component attribute of an instance specifies the component being instantiated. The value is
a string used to find implementations for this instance, by searching in the available artifact
libraries. It is the name assigned by the component developer to the component specification used
as the basis for implementations. Component specifications are themselves XML documents/elements
called OCS (OpenCPI Component Specification). They have names (used to match this attribute’s
value), and describe the ports and properties that apply to all implementations of that component.
This attribute is required, and answers the question: what function should this instance perform? The process by which OpenCPI searches for implementations based on this attribute is described above in the package attribute section.
This attribute may have a package prefix (ending in a period) to indicate which package contains the
component specification indicated. If there is no prefix (and no periods), the package prefix is
taken from the default for the whole assembly, which is specified using the package attribute of
the top-level application element. The component attribute value can have a leading period,
which indicates a component name relative to the prevailing package prefix.
This attribute provides the PackageID of the component by combining any prefix from the
application’s package attribute with its value if the value contains no periods. Package IDs are
meant to be global identifiers constructed similar to Java package names. The three cases of this
attribute are:
No periods, simply appended to the prevailing package prefix.
Leading period, also appended to the prevailing package prefix.
Periods but not leading periods, used by itself with no prefix applied,
The second case is useful to indicate components in component libraries in the same project, e.g.
component=".mylib.mycomp".
Name attribute (optional)
The name attribute of an instance is optional, and provides a unique identifier for the instance
within the application. If it is not supplied, one is assigned to the instance. If there is only one
instance in the application for a type of component (i.e. the component is used only once), the
assigned instance name is the same as the component name (without package prefix). If more than one
such instance (of the same component) exists in the application, the assigned name is the component
name (without package prefix) followed by the decimal ordinal of that instance among all those for
the same component. Such ordinals are assigned starting with 0.
For example, in the example application above, there is one instance of the file_read component,
and thus its instance name would be file_read. If the application used file_read twice (e.g.
two different components were taking data from different files), the two instances would be named
file_read0 and file_read1, in the order they occurred in the OAS.
Connect attribute (optional)
Connecting instances together in an assembly can be done one of two ways. Using the connect
attribute of an instance is the simplest, but cannot express all connections. The connection
child element of the application element can be used to express all types of connections. It is
described later.
The connect attribute defines exactly one connection from an output port of this instance to an
input port of another instance. Its value is the name of the other instance. If this instance only
has one output port and the other instance only has only one input port, then these are implied.
The optional from attribute specifies the name of the output port of this instance if needed
(if there are more than one), and the optional to attribute specifies the name of the input port
of the other instance (if there are more than one). An example using all three attributes is:
<application>
<instance component="psd"
connect='demod' from='myout' to='demod_in'/>
<instance component="demod" selection='model=="rcc"'/>
</application>
This simple connection method is useful for the many components that have only one output port.
Selection attribute (optional)
This attribute optionally specifies how to choose among alternative implementations when more than one is available. This capability also provides a way for the application to specify minimum conditions on the candidate implementations found in the library.
The attribute value is an expression in the syntax of the C language, with all the normal operators,
including the ?: ternary operator. Logical expressions (e.g. “a == 1”) return 1 on true and
when false. The variables that may appear in the expression are either:
Property names that have fixed (not runtime variable) values
Built-in identifiers that indicate well-known attributes of the implementation. The built-in identifiers are:
model: the name of the authoring model of the implementation, e.g.rcc.platform: the name of the platform the implementation is built for, e.g.rocky9orzedos: the name of the operating system the implementation is built for, e.g.linux
The value of the selection expression is considered an unsigned number, where a higher number is better than a lower number, and zero is considered unacceptable. I.e. if the expression when evaluated for an implementation has a zero value, that implementation is not considered a candidate. A simple example might be:
model=="rcc"
This indicates that the model must be rcc since otherwise the expression’s value will be zero.
The example:
error_rate < 5 ? 2 : 1
indicates that the error_rate property is relevant, and if less than 5, it is better than when
greater than or equal to 5, but the latter is still acceptable.
If there is no selection expression, the “score” of the implementation is 1, unless it has hard-wired connections to other collocated workers (e.g. on an FPGA). In this case its value is 2.
An example of using the selection attribute is:
<application>
<instance component="psd" selection='latency < 5'/>
<instance component="demod" selection='model=="rcc"'/>
</application>
It indicates that the psd instance needs an implementation with latency less than 5, and the
demod instance must have an implementation with an authoring model of rcc.
From attribute (optional)
This attribute is used to specify the name of the output port of this component instance in
conjunction with using the connect attribute described above.
To attribute (optional)
This attribute is used to specify the name of the input port of the other component instance in
conjunction with using the connect attribute described above.
External attribute (optional)
This string attribute is used to specify a port of the instance that is to be considered an external
port of the entire application. Its value is the name of this instance’s port that should be
externalized. The external application-level name of the port is the same as its own name on this
instance. To specify a different name, use the connection element described below. Note that the
name of this attribute is singular and distinct from the externals attribute described next.
Externals attribute (optional)
This boolean attribute is used to specify that all unconnected ports of the instance are to be
considered external ports of the entire application. The external application-level names of the
ports are the same as their own name on this instance. To specify different names, use the
connection element described below. Note that the name of this attribute is plural and distinct
from the external attribute described previously.
Worker attribute (optional)
This string attribute is used to specify a particular worker to use for this instance. Usage of this attribute is rare and generally not recommended in the OAS since it bypasses the automatic selection algorithm for choosing the worker based on available implementations and available containers. The string value is the name of the worker, with or without package prefix and with or without any suffix for “authoring model”.
Slave attribute (optional, deprecated)
This string attribute specifies the name of another instance in this application that acts as a
slave to this instance. This attribute implies that any worker used for this component must be a
proxy and that the worker used for the identified slave instance must be an appropriate slave worker
for the worker chosen for this proxy. This attribute is used when there is a single slave for this
instance. If there are multiple slaves for the instance, the slave element described in
Slave Elements must be
used. More information about slaves and proxies is in that section. This attribute is deprecated in
favor of the default automatic inclusion of slave workers in the application when a proxy instance
is specified in the application.
Buffersize attribute (optional)
This numeric attribute specifies the size of message buffers for the connection on the output port
in conjunction with using the connect attribute described above. For more details on message
buffers, see the description of the connection element.
Property Elements within the Instance Element (optional)
The <property> element is used as a child of the <instance> element to specify configuration
property values that should be configured in the worker when the application is run, prior to the
application being started. If no <property> element is specified for a property for the
instance, the default value for the property (from the component’s OCS) is implicitly being
requested by the application. Within an instance element, some examples of property (child)
elements are:
<instance component="psd">
<property name="size" value="17"/>
<property name="symmetric" value="true"/>
</instance>
Properties can only be set if they are specified in the spec (OCS) with access as initial or writable. Properties labeled in the OCS as initial can only be assigned values before the application runs, while those labeled as writable can be specified before the application runs (like initial properties) but can also be changed during execution, dynamically.
Property values may also be set outside the OAS (on the command line or as a runtime-parameter). Using these mechanisms outside the OAS makes the OAS more reusable since it can be used with different property values settings.
Name attribute (required)
The name attribute of a property element must match the name of a property of the specified
component. I.e., it must be one of the defined configuration properties of the component. Component
specifications define properties that are common to all implementations of a component. Component
implementations (workers) can also define additional properties that are specific to that
implementation, but mentioning such properties will only be accepted if the selected implementation
has them. Otherwise an error results.
Value attribute (one of value or valueFile is required)
The value attribute is the value to be assigned to the configuration property of the worker just
before being started. The attribute’s value must be consistent with the data type of the property in
the component specification. E.g. if the type of the property is ulong, then the attribute’s
value must be numeric and not negative.
The complete syntax of property values is described in the Property Value Syntax and Ranges section.
ValueFile attribute (one of value or valueFile is required)
The ValueFile attribute is the name of a file containing the value to assign to the property.
Using this attribute, rather then the value attribute, is convenient when the value is large,
such as when the property’s value is an arrays of values. When valueFile is used, all new lines
in the file are interpreted as commas.
The complete syntax of property values is described in the Property Value Syntax and Ranges section.
DumpFile attribute (optional)
The DumpFile attribute is the name of a file into which the value of the property will be
written after execution. When DumpFile is used, all commas in the value are replaced by new
lines in the written file. The complete syntax of property values is described in the
Property Value Syntax and Ranges section.
Delay attribute (optional)
Normally property values are set before the application starts, but it is possible for a property
value to be set after the application starts. This delay attribute specifies when the
property value should be set after the application starts. A value of zero means as soon as
possible after the application starts. Any value greater than zero indicates the amount of time to
wait (in seconds) after the application starts, before setting the value. The delay values are in
units of seconds, and may be floating point. Thus 5e-6 would indicate 5 microseconds.
Multiple delays and values may be specified for a property to indicate a timed sequence of property
settings. This is done by inserting multiple set elements as child elements of the property
element. Each set element may have its own delay and indicate the value to be set using the
value or valuefile attributes. Here is an example that sets a property’s value 5 seconds
after the application starts, and then sets a different value 10 seconds after the application
starts.
<application>
<instance component='mycomp'>
<property name='control'>
<set value='123' delay='5'/>
<set value='345' delay='10'/>
</property>
<property name='other' delay='20' value='789'/>
</instance>
</application>
Slave Elements within the Instance Element (optional, deprecated)
This element is used to specify one or more slave instances that act as slaves to this instance.
This element implies that any worker used for this instance must be a proxy and that the worker used
for the identified slave instance must be an appropriate slave worker for the worker chosen for this
proxy instance. Each slave element must have a name attribute whose value is the name of the
other instance in the assembly that will act as a slave to this instance.
Workers can have a proxy/slave relationship where one worker (a proxy) is allowed to control other workers (slaves). A proxy worker is designed and coded to control specific other workers as slaves, and this worker-to-worker relationship is declared by the proxy worker. Workers identified as slaves by proxies have no special declaration themselves and they do not need a proxy worker to function properly. This relationship is used by the code in the proxy worker
An example of using the elements is:
<instance component="device1"/>
<instance component="device2"/>
<instance component="device_controller">
<slave name="device1"/>
<slave name="device2"/>
</instance>
This slave element exists to allow slaves to be parameterized and connected within the application. This element is now deprecated. Now, the slaves will be included in the application automatically if a worker used for an instance is a proxy.
Property Elements within the Application Element (optional)
Property elements at the top level of an application (rather than under an instance element),
represent properties of the application as a whole. They are essentially a mapping from a top-level
property name to a property of some instance in the assembly.
This provides a convenient way to expose properties to the user of an application without requiring them to know the internal structure of the application. For example:
<application>
<property name='infile' instance='file_read' property='filename'/>
<instance component='file_read' connect='add1'>
<property name='filename' value='test.input'/>
</instance>
<instance component='add1' connect='file_write'/>
<instance component='file_write'>
<property name='filename' value='test.outputwrong'/>
</instance>
</application>
The above example provides an application-level property named infile, which is mapped to the
filename property of the file_read component instance. In addition to the attributes listed
here, the top-level property elements also accept the value, valueFile, and dumpFile
attributes described in the previous section.
Name attribute (required)
The name attribute of an application-level property is the name that users of the application
will use to read, write or display the value. If the property attribute just below is not
present, then this name is also the name of the instance’s property.
Instance attribute (required)
This attribute specifies the name of the instance that actually implements this property for the application. It is the instance that the application-level property is “mapped to”.
Property attribute (optional)
If the application-level name of this property is not the same as the instance’s property to which is it mapped, this attribute is used to specify the actual property of the instance. It is a string property that must match a property of the instance.
Connection Elements within the Application Element (optional)
Connecting instances together in an assembly can be done one of two ways. Using the connect
attribute of an instance is the simplest (described above), but cannot express all connections.
The connection child element of the application element can be used to express all types
of connections. It describes connections among ports and also with “the outside world”, i.e.
external to the application. The connection element has optional name and transport
attributes, and port and external child elements.
An example of an application with some connections is:
<application done='file_write'>
<instance component='file_read'/>
<instance component='bias'/>
<instance component='file_write'/>
<connection transport="socket">
<port instance='file_read' name='out'/>
<port instance='bias' name='in'/>
</connection>
<connection>
<port instance='bias' name='out'/>
<port instance='file_write' name='in'/>
</connection>
</application>
The first connection connects the out port of the file_read instance to the in port of
the bias instance, and specifies that the connection should use the socket transport
mechanism. The second simply connects the out port of the bias instance to the in port
of the file_write instance. This second connection could have been more simply accomplished by
using the connect attribute on the bias instance.
Name attribute (optional)
This attribute specifies the name of the connection. It is only used for documentation and display
purposes and has no specific other function. If it is not present, a name is assigned according to
the conn<n> pattern, where <n> is the number of the connection in the application
(0 origin). If the connection is thought of as a “wire”, this is the name of the wire that is
attached to various other things (instance ports and external ports). For an example of external
ports, see OA::Application::getPort Method.
Transport attribute (optional)
This attribute specifies what transport mechanism should be used for this connection. OpenCPI supports a variety of transport technologies and middlewares that convey data/messages from one instance’s port to another. Normally the transport mechanism is chosen automatically based on which ones are available and optimal. This attribute allows the application to override the default transport mechanism and force the usage of a particular one. The ones supported at the time of this document update are:
Name |
Description |
|
Programmed I/O using shared memory buffers between processes |
|
DMA or PIO over the PCI Express bus/fabric |
|
RDMA using the OFED software stack, usually for Infiniband |
|
RDMA using TCP/IP sockets |
|
RDMA using Ethernet (link layer) frames |
|
RDMA using UDP/IP |
Some of these transport mechanisms are only available if specifically installed in a system.
Buffersize attribute (optional)
This attribute specifies the message buffer size for the connection. Connections convey messages from input ports to output ports, using message buffers. The buffers must be large enough to fit the messages. The exact mechanism for buffering is determined by the low-level mechanisms and technologies used to convey the messages. Normally the OpenCPI runtime framework determines buffer sizes based on a combination of the protocol used for the connection and other information supplied by workers.
When there is no protocol associated with any of the ports of the connection, and no worker-based information to determine the buffer size, a system-wide default buffer size is used, which is 2K bytes.
If the application wants to override the above determination of buffer size, perhaps to reduce latency (by reducing buffer size) or increase throughput (by increasing buffer size), it can use this attribute to do so. Even when this attribute is specified, this value can be overridden by command line arguments or runtime ACI values. In some cases buffer sizes are constrained by hardware limitations.
Port Elements within the Connection Element (optional)
This element is used to specify a port that this connection should be attached to. The most common use of this element is to specify the consumer and producer of the connection, using a port element for each, within the same connection element.
Instance attribute (required)
This attribute specifies the name of the instance of the port to be attached to this connection. This instance name is used along with the name attribute to specify the port.
Name attribute (required)
This attribute specifies the name of the port that should be attached to this connection. This port name is scoped to the instance defined in the instance attribute of the connection element.
BufferCount attribute (optional)
This attribute specifies the number of message buffers to use at this port. Increasing this value can sometimes improve throughput at the expense of using more memory.