Developing OpenCPI Assets in Projects

In OpenCPI a project represents a work area in which a variety of assets are created and developed. Projects can contain all types of assets that are described fully either in this document or in others. A project can contain:

• Component libraries with specs and workers.

• Applications (described in the OpenCPI Application Development Guide).

• HDL primitives and assemblies (described in the OpenCPI HDL Development Guide)

• HDL devices, cards, slots (described in the OpenCPI HDL Development Guide)

• Platform support assets (described in the OpenCPI Platform Development Guide)

A project is a standard directory structure that holds the various OpenCPI assets in both source code form and built form, along with the XML files that describe how they are built. The project structure provides a means to bundle a collection of assets which may have a logical relationship or be created for a specific application.

The ability to develop assets outside of a project (a.k.a. standalone) is also supported, but is preliminary and subject to change, and not discussed further in this section.

The ocpidev create command is used to create and then populate a project directory structure with the various asset types. The created skeleton directory structure is always buildable.

The structure of a project, and types of assets (shown enclosed in <>), is shown in the following diagram (with the other files omitted except at the project level).

Project.xml
Project.exports
applications/<applicationXYZ>/
specs/
components/<componentlibXYZ>/<workerXYZ>/
                            /specs/
---OR---
components/<workerXYZ>/
          /specs/
hdl/primitives/<primitiveXYZ>/
hdl/assemblies/<assemblyXYZ>/
hdl/platforms/<platformXYZ>/
hdl/devices/<device-workerXYZ>/
           /specs/
hdl/cards/<card-device-worker>/
         /specs/

The optional top level specs directory is separate from the specs directory in any component library. It is a project-wide specs directory that is usable by all component libraries in the project. It can exist in the project, for use by other projects, even if there are no component libraries in the project.

Creation of a project (using ocpidev create project) creates a skeleton directory structure that is buildable, but it will build nothing initially as it contains no assets. All the intermediate directories are created by ocpidev as needed. If there are any component libraries created in the project (using ocpidev create library) a components directory is created, under which those component libraries will be created. Alternatively, for simpler projects which only have a single component library, the components directory is the single component library (created using ocpidev create library) and workers are created directly under components. A project currently cannot easily be changed between the “flat”, single library structure and the “hierarchical”, multiple library structure.

In addition to the various directories, two required files are generated at the top level when the project is created by ocpidev create project:

Project.xml

the XML file that specifies attributes that are used project-wide, for all assets at all directory levels.

Project.exports

a file that specifies which assets and files should be visible from outside the project, i.e. visible to other projects which use some of the assets in this project.

These files are automatically created when the project is created, but may be edited later as necessary. The Project.xml file must exist; the Project.exports is optional and created and edited manually.

Managing Project Assets

The ocpidev tool described in detail later is used to manage all the asset types in a project. It is used to create, delete or test assets. Once created, the assets are based on text files that must be edited. Assets are created using the ocpidev create command and they are deleted using the ocpidev delete command. Most assets, including projects themselves, are based on a directory, with an XML file for descriptive information in that directory. These include:

• Component libraries

• Workers

• Unit test suites

• HDL device workers

• HDL platforms

• HDL primitives (cores and libraries)

• HDL assemblies

• Applications (except the simplest ones)

When an asset is created, the appropriate directories are also created, an initial XML file is created in the new directory, and in some cases other initial files are also created.

Some assets are simply files and when created, an initial version of the file is created in the appropriate directory in the project. This type of asset includes:

• Component Specs

• Protocol Specs

• HDL card definitions

• HDL slot definitions

• Applications (for simple XML based applications with no ancillary files)

When creating specs, protocols, workers, and unit tests, a library option (-l <library>) may be supplied to ocpidev create, indicating which library the asset should be added to.

When adding a platform-specific device worker or device proxy, a platform option (-P <platform>) may be supplied to indicate which platform-specific device library to add the device worker to. Portable device workers that are not platform-specific should not use this option.

Package IDs

A package-ID is the globally unique identifier of an OpenCPI asset. A project’s package-ID is used when it is depended on by other projects. A component’s package-ID is used to reference it in applications or workers. While all assets have package-IDs (either explicitly specified or inferred from the directory structure), only certain assets are actually identified by their package-IDs in the current OpenCPI release. Eventually, package-IDs will be used uniformly and universally.

A package-ID is a hierarchical sequence of names separated by periods. This OpenCPI package naming scheme follows the Java package naming conventions, which are roughly based on a reversed Internet domain name with the top-level domain first, and the more specific or local names after that. Thus if CNN had an OpenCPI project full of news-related components, its package-ID might be com.cnn.news.

Warning: When changing the PackageName or PackagePrefix of an existing project the project needs to be both unregistered and re-registered then cleaned and rebuilt. This also includes cleaning and rebuilding any projects that depend on this project.

Package-ID Usage and Structure

Package-IDs are divided into two parts, the package-prefix and the package-name. The package-prefix is the string before the last period, and the package-name is the name after the last period. Generally the package-ID of an OpenCPI asset consists of the package-prefix from its containing environment (e.g. the project of a library, or the library of a worker). Its package-name is simply the name given when the asset was created (using ocpidev create). Thus the package-prefix acts as the name scope for the asset. This package naming scheme (from Java) provides for globally unique identifiers that are human readable (and type-able) and already administered by a globally known organization. A package-ID consists of <package-prefix>.<package-name>.

OpenCPI defines two reserved top level package prefixes: ocpi and local. The ocpi prefix is managed by the OpenCPI maintainers and is used for assets that are located at the OpenCPI github repository. Projects that are maintained there all have names with this prefix (currently the three projects in use are ocpi.core, ocpi.assets, and ocpi.inactive). The second reserved prefix (local) is the default packageID for all projects that are created with no explicit package-ID. This means that basic development projects (or tutorial examples) are not required to have a global identity to simply get started. As soon as other projects need to use or depend on a project, it should have a more explicitly assigned package-ID.

The package-ID of a component specification is generally prefixed with the package-ID of its containing library or project (followed by a period). This prefix is the name scope in which the component is defined. This allows components to be specified and implemented by different organizations, while still allowing any implementation found in a library to satisfy any (other) organization’s component specifications. E.g., my project can have an additional, alternative implementation of a component specified in another organization’s library, or can define its own specification for a component with the same package-name with a different package-prefix (name scope).

Package-ID attributes in XML Files

There are several cases where attributes are set in XML files to help specify package-IDs. In an application XML file, the top level package attribute provides a default package-prefix for all components mentioned in the file (that do not have periods in their name).

A component spec XML file can specify a package-prefix in a top-level package attribute, This would override the default, which is the package-ID of the library in which the spec is defined. If the spec is not in the library, but rather in the top level specs directory of a project, this would override the default prefix from the project’s package-ID.

The package-ID of an asset is normally inferred from the name it was created with serving as package-name with the package-ID of its containing asset serving as package-prefix. When this inferred package-ID is not what is needed, certain attributes in the asset’s XML file can override this default behavior. The PackagePrefix attribute can override the default package-prefix supplied from the package-ID of the containing asset.

The default package-prefix for projects is local. As soon as this default is not what is wanted, the PackagePrefix attribute can be set in the project’s Project.xml file in the top level project directory. Similarly, a component library in a project could override its prefix (which would normally be the package-ID of its project) using the PackagePrefix attribute in the library’s <library-name>.xml file.

In the unusual case where the package-name should be different than the name the asset was created with (which is also the name of its directory), the PackageName attribute can be used to override that part of the package-ID.

When it is required to simply set the entire package-ID for an asset, the PackageID attribute can be used. This overrides any logic for combining the package-prefix and the package-name.

How to Determine an Asset’s Parent (and thus Default Package-Prefix)

The parent of a component library, any of:

components/
components/<library>
hdl/devices/
hdl/cards/
hdl/adapters/
hdl/platforms

is the project itself.

An HDL platform’s parent is the hdl/platforms directory.

A hdl/platforms/<platform>/devices library’s parent is the containing platform.

A project has no parent, and so its package-prefix defaults to local.

The default package-prefix of a component (which does not have its own directory) is <parent's package-ID>.<component-name>.

Package-ID Examples

Within the ocpi.assets project the Project.xml file contains:

PackagePrefix='ocpi'

Thus the package-ID for the project is: ocpi.assets.

The dsp_comps library in the same project (located at assets/components/dsp_comps) has no attributes set in its <library‑name>.xml file, so:

• The package-prefix defaults to package-ID of parent (project): ocpi.assets

• The package-name defaults to library’s directory name: dsp_comps

• Thus the package-ID is: ocpi.assets.dsp_comps

• If we had PackageID='full_package' in the file dsp_comps/dsp_comps.xml, the package-ID of the library would be, entirely, full_package

The iq_imbalance_fixer component spec in assets project’s dsp_comps library would be referenced in an application as:

ocpi.assets.dsp_comps.iq_imbalance_fixer

This is because a spec’s parent is the library it is in.

The Project Registry: How Projects Depend on and Find Each Other

Every project depends on some other projects. At a minimum all projects depend on the core project provided as part of OpenCPI. Its package-ID is ocpi.core.

When assets in a project depend on assets in other projects, the project must declare that it depends on such other projects. Here are some examples of how assets can depend on other assets, and thus how an asset in one project may depend on assets in other projects.

• A worker may depend on a spec (OCS), whether in the same library as the worker, a different library in the same project or in a different project.

• An HDL assembly may depend on workers in any library in the same project or workers from libraries in other projects.

• An HDL worker may depend on an HDL primitive library in the same project or in a different project.

A project declares its dependency on another project by either specifying the dependency when the project is created (using ocpidev create project), or adding the dependency to its Project.xml file later. In either case the dependency is declared by providing the package-ID of the other project. As mentioned above, the dependency on the ocpi.core project is always assumed.

A project is visible to other projects by being registered under its package-ID. A project can only depend on another if that other project is registered with the OpenCPI installation. The default condition of a project, when created, is to be unregistered and thus not visible to others. When a project is in a suitable condition to be depended on by others (i.e. it contains assets that are ready to be used by other projects), it can be registered at that time (using the ocpidev register command). This action can be reversed using the ocpidev unregister command.

The registry includes a mapping between package-IDs and the pathname of registered projects.

Showing Registered OpenCPI Projects

ocpidev show projects

Show all OpenCPI Projects in the current registry.

Usage:

$ ocpidev show projects [OPTIONS] [Project]

Arguments:

• [Project]

Options:

• --format [json|table|simple]: Specify the format to display the asset(s). [default: table]

• --help: Show this message and exit.

The built-in projects supplied with OpenCPI are automatically registered in the default project registry as part of the installation process.

Registering an OpenCPI Project

ocpidev register project

Register an OpenCPI Project.

Usage:

$ ocpidev register project [OPTIONS] [NAME]

Arguments:

• [NAME]: [default: (dynamic)]

Options:

• -r, --registry-dir PATH: Specify directory containing registry. [default: /home/user/opencpi/project-registry]

• --help: Show this message and exit.

Unregister an OpenCPI Project

ocpidev unregister project

Unregister an OpenCPI Project.

Usage:

$ ocpidev unregister project [OPTIONS] [Project]

Arguments:

• [Project]

Options:

• -f, --force

• --help: Show this message and exit.

Unregistering a project does not delete the project. It only removes it from the registry. You can also create and register a project with a single ocpidev command:

$ ocpidev create project –-register <project-name>

Projects That Implement Platforms

While platform assets (sometimes called platform support packages) are one of many types of OpenCPI assets, they are special in two ways:

• they enable other assets to be built (compiled, synthesized, etc.) targeting their platform

• when they are in a registered project, they are usable for asset building for all other projects regardless of whether those other projects declare a dependency on the project containing the platform asset

A project A does not have to explicitly depend on project B simply to build A’s assets targeting a platform implemented in project B. This allows building for platforms without knowing which project they are implemented in. For any other dependency between assets in one project and another, the project dependency must be declared when the project is created or later in the project’s Project.xml file.

Project Registries for Sandboxed Project Development

Projects depend on other projects, and make themselves visible to other projects via a project-registry that is part of the OpenCPI installation. The registry includes a mapping between package-IDs and the pathname of registered projects. This mapping is shown using the ocpidev show projects command.

There are two different relationships between projects and registries:

• Every project is automatically associated with a project registry upon creation and thus can view and depend on other projects registered there.

• A project can be registered in its associated project registry, and thus be visible to other projects associated there. Projects are not initially registered.

XML Files in Projects

Most of the directories in a project contain an XML file containing attributes pertaining to the directory and those below it. The name of the file is the name of the directory, minus any suffix in the directory name, plus the .xml suffix. E.g. for a directory named myworker.rcc, the name of the XML file in that directory is myworker.xml. For directories that are assets, the name is the name of the asset (plus .xml). For directories that are just containers for assets, the name is still the name of the directory, e.g. in the applications/ directory in a project, there can be an XML file named applications.xml. There are two exceptions to these naming rules:

• the XML file for a project has the fixed name Project.xml, allowing the project to be renamed or moved without changing the name of the file.

• the XML file for a unit test suite, whose directory name is <component‑name>.test, is <component-name>‑test.xml.

The ocpidev create command creates the XML file for the asset, even when the asset does not have a directory (such as a protocol definition XML file).

When a directory contains a number of subdirectories for the same type of asset, an attribute can be set in its XML file which lists the assets to build, or the assets to exclude from building. These attributes are optional, and when not specified, all such assets are built. For example, in a component library where all the subdirectories contain workers, the default XML file contains no attributes listing workers, which implies that all worker subdirectories should always be built. If there are workers that should not be built, or they should be built in a particular order, then the Workers attribute can be specified to list the explicit set of workers that should be built, in order, e.g.:

<library Workers='fft.rcc fft.hdl'/>

Alternatively, if there are a few workers that should not be built, the ExcludeWorkers attribute can list them, in which case all workers except those listed will be built.

This same idea applies to directories in a project that contain HDL assemblies, HDL platforms, HDL primitives, applications, devices, cards, and slots, etc. The exact name of this attribute, and other optional attributes, are described in the section for each asset.

Some attributes set in the Project.xml file in the top-level project directory apply anywhere in a project, when ocpidev build is invoked in any of the project’s directories.

The Project.xml File for Project-wide Settings

This XML file is required in the top-level directory of a project. It contains attribute settings that apply to all levels of a project. Its existence indicates that the directory is in fact a project. In all directories under a project, this file is found by looking in parent directories until the Project.xml file is found. This is similar to how the git tool finds the top level of a git repository by searching for a directory named .git.

The build-related attributes in this top-level project file or a similar file in subdirectories with assets under them, are applied to building lower-level assets automatically. An example is the ComponentLibraries attribute.

Within an OpenCPI project, the ocpidev build or ocpidev clean command can be run directly in any asset’s directory (without specifying a noun or asset name), or in any of the “plural” directories containing similar assets.

The top-level element name for the Project.xml file is project. For example, here is the Project.xml file for the built-in “core” project (ocpi.core):

<project PackageName='core' PackagePrefix='ocpi'/>

The following table shows the project attributes that may be set in a Project.xml file. They are all optional. The Project.xml file must be present, but may be empty.

Table 30: Attributes Set in the Project.xml File

Attribute Name in Project.xml	Default	Description
PackageName	Project directory’s name	The name used for this project in its package-ID
PackagePrefix	local	The package-prefix for this project’s package-ID. The default, local, is appropriate when the assets are intended to be used only in the local organization or prior to registration.
PackageID	<prefix>.<name>	The package-ID of the project, overriding PackageName and PackagePrefix
ProjectDependencies	""	A list of the package-IDs of registered projects that this project depends on. The ocpi.core built-in project is assumed.

In order to avoid name space collisions when using multiple projects or component libraries (e.g. spec names and worker names), the project’s package-ID is the default namespace for all named assets in the project.

The ProjectDependencies attribute should be used to declare other projects that this project depends on. This will cause these other projects to be used when searching for assets that are subject to search paths, such as:

• OCS files (when building workers)

• HDL primitives (when building HDL workers)

• Component libraries when building workers (for slaves of proxies, or devices of emulators)

While project dependencies should generally be the package-IDs of registered projects, they may also be relative or absolute pathnames of other projects. However using such direct pathnames is deprecated and may not be supported in the future.

Other useful XML attributes that can be specified in the Project.xml file include attributes providing default lists of build targets and platforms:

HdlTargets
HdlPlatforms
RccPlatforms
RccHdlPlatforms

Search list attributes may also be put in this file, although care should be taken to not unduly pollute search spaces for all assets in the project. These attributes include:

ComponentLibraries
HdlLibraries

Such attributes may be better placed in the <library-name>.xml files in component library directories.

Project Exports

When a project’s assets are used by assets outside the project, they use the project’s assets via its exports. Exports of a project are the files within the project that are explicitly made visible and usable from outside the project. I.e., without exports, nothing in a project is intended to be visible outside the project. A project’s complete directory structure contains source files and artifacts of the build process. The exports are the files needed by users of the project, and can be thought of as the installable and deliverable subset of the files in the project after it is built.

When other projects depend on a project, as specified in those projects’ Project.xml file ProjectDependencies setting, that means they use project exports from those projects they depend on. The projects they depend on must be registered.

The exports tree is a directory containing the project’s exports, and is constructed as a tree of symbolic links, under the directory named exports at the top level of a project. The structure of the exports tree is not necessarily the directory structure of the project itself, but is a structure appropriate and convenient for users of the project’s assets. By constructing the exports tree using symbolic links, the exported view of a project uses no extra space (no copies). The assets are therefore used externally exactly where they exist in source form or where they are built (although indirectly via the symbolic links in the exports tree).

Much like there is a standard directory structure for OpenCPI projects, there is an implied standard exports tree based on the contents of a project. This directory, exports/, is updated when builds are done at the project level.

The exports tree has two different uses. One is to allow the project’s intended deliverable results to be used in-place, without any copying or “installing”. The other is to provide an implicit recipe or bill-of-materials for creating an installable binary package for users of the project. In this latter case a simple single file deliverable package can be created (see below).

The Exports Tree

The top level directory exports, is created and populated automatically based on the file Project.exports. This exports/ directory can always be deleted and recreated. It is never manually constructed or changed. If the Project.exports file is empty or does not exist no exports are created. The default export tree is created based on the assets in the project. It is the set of exports when the Project.exports contains the single line containing the single word: all.

The next sections describe the default exports tree and the format of the Project.exports file that can be used to add or subtract from the default exports.

Here are the rules used to populate the default exports tree when project-level builds are invoked at the top level of a project and the Project.exports file contains the single line: all.

For an example of how the exports tree works:

Note: This structure is subject to change between Major releases of OpenCPI and should not be depended on by any end user developed code.

Component library deliverables are made available in the exports tree under the lib directory, using the name of the library. If there is a component library in the project in the directory components/util_comps, where its own locally built deliverables are in its lib subdirectory (components/util_comps/lib), then these deliverables are available in the exports tree using lib/util_comps, which is a symbolic link to components/util_comps/lib, i.e.:

exports/lib/util_comps -> ../../../components/util_comps/lib

This directory structure is subject to change but creates an export view of the deliverables of the project using a sparse set of symbolic links. Users of the project, seeing only the exports tree, see lib/util_comps for this library.

All assets have a default place in the exports tree where other projects that depend on them can find them, if the project owner decides to export them.

The Project.exports File

The Project.exports file specifies which assets and files in the project should be made visible, usable, and accessible from outside the project (usually by other projects). If the file is missing or empty (or only contains blank lines or # comments), the project is not intended to be used by others outside the project.

This nothing-is-exported condition is appropriate for projects in development before anything is ready for use by others (and before a package-ID is assigned). It is also appropriate for projects that contain end products like applications for test purposes.

At the other end of the “visibility” spectrum, the Project.exports file can contain a single line containing the word all, which indicates that all assets in the project should be visible/usable from outside the project.

Between these two extremes we can selectively export assets and/or files in the project.

Exporting Assets by Type or Name

To export assets, we simply provide their type and name, or in some cases the plural form of their type to indicate all assets of that type. Here is a list of the lines you can use to export assets by listing their types and names. When <name> is missing, it indicates that all the assets of the indicated type should be exported.

all
spec <name>         # a spec in the top level specs/ directory
specs               # all specs in the top level specs/ directory
libraries           # all component libraries under "components"
library <name>      # <name> can also be a path within the project
hdl platforms
hdl platform <name>
platforms           # all possible platforms
rcc platforms
rcc platform <name>
hdl primitives
hdl primitive library <name>
hdl primitive core <name>
hdl assemblies
hdl devices

Exporting Individual Files and Directories

The Project.exports file may also contain lines that add and subtract files from the asset exports of a project. This capability it rarely used but covers some edge cases to augment or prune the exports tree created based on asset exports. File additions are lines that start with a plus sign (+), and subtractions are lines that start with a minus sign (-). White space (spaces or tabs) can precede the -, +, or # characters.

The format of addition (+) lines is two fields separated by white space. The first field is the relative pathname within the project for the file to be exported, and the second field is the location in the export tree where the file should be linked. If the second field ends in a slash, then the filename part of the first field is used as the file name in the exports tree. Pathnames or other names with embedded spaces are not currently supported.

The line:

+special_dir/special_file include/ # this exports my special file

would export the file in the project named special_dir/special_file as:

include/special_file

If the name of the exported file should be different, it can be included in the second field, e.g.

+special_dir/special_file include/different-file

If the second field is blank (doesn’t exist), then the project file or directory is exported in the same place is it exists,

+special_dir # export this directory where it is in the project

would simply make special_dir a top level directory in the exports tree.

Any directory that is exported implicitly exports all files underneath it.

The first field can also have wildcard patterns using normal sh/bash wildcard patterns. A special string, <target>, indicates the software target that is currently being exported. When a project is exported, it is done in the context of a particular software target. The line below

+applications/myapp/target-<target>/myspecial_exe bin/<target>/

exports a secondary executable in the application.

Subtraction lines start with a minus sign (-). As with addition lines, the first field is the pathname within the project, with possible wildcards and <target> strings.

In both additions and subtractions the first field can actually be an extended regular expression if the field starts with a vertical bar (|). For example, an addition line might be:

+|^(abc|xyz) special_dir/

which would export anything starting with abc or xyz in the exported directory special_dir.

Using Other Projects that Exist Outside the Project Being Developed

The convenient and recommended way to use a project A from another project B is to add the package-ID of A to the ProjectDependencies attribute in the project B’s Project.xml file. This declares the dependency without using any environment settings or pathnames. The dependency is persistent (in Project.xml) and stays with the project even if it is copied or moved. Since the other project is specified in ProjectDependencies by its package-ID, it can also be moved (and re-registered) and the dependency is still valid.

The actual directories searched within a project depends on the type of asset being sought. E.g., if a component library is being sought, the search will look for that component library within all projects mentioned in ProjectDependencies. E.g.:

ProjectDependencies='local.utils ocpi.assets'

would indicate that this project depends first on the local.utils project and then also depends on assets in the ocpi.assets project (as well as ocpi.core which is added to the dependency list). Any component library being sought might be in either one, and if found in local.utils, the ocpi.assets project will not be used.

When a more dynamic setting is needed, e.g. for temporarily using one version of a project vs. another, the OCPI_PROJECT_PATH environment variable can be used. This variable specifies a colon-separated set of other projects (by pathname) to be searched in order. These are searched before projects mentioned in the ProjectDependencies attribute to allow the environment variable to temporarily override what is declared persistently in the project. Projects in OCPI_PROJECT_PATH, are examined by looking in their exports subdirectory if it exists.

To summarize searching at the project level, the order is:

1. Use assets in the local project.

2. Use assets in projects in OCPI_PROJECT_PATH to temporarily override anything later in this list.

3. Use assets in ProjectDependencies, in the order given in that attribute (with ocpi.core automatically added at the end, as previously mentioned)