2. Architecture

This architecture guide is the home for vision, requirements (scenarios, use cases, functional and non-functional requirements) as well as of architecture stuff.

Furthermore, the structure is rather arbitrary, established on the fly rather than a “formal” requirement specification or architecture documentation

Nevertheless, it is the home to understand some backgrounds, ideas better.

Note

This section does not claim to be up to date.

2.1. Preample

Yocto builds require a complex environment to run on.

Application developers, BSP developers, integration engineers, release engineers have different needs to build something with Yocto.

A lot of upstream input has to be maintained in terms of

  • bblayers
  • sources

Different quality attributes are applied like

  • fast turn around time for app developers
  • reproducibility for release engineers
  • fast feedback for integrators

The output of Yocto builds is complex as well and is handle differently.

2.1.1. Build Process

  1. Access the build host (login to bare metal machine or pick docker image)
  2. Gather the build specification (layers, recipes, configuration)
  3. Initialize the build environment (source an init script)
  4. Pick a machine and an image/recipe and build
  5. Use the build result (publish, run)

2.1.1.1. Gather build specification

Quite often upstream projects suggest using the Google repotool to access bblayers available and maintained in git repositories.

Alternatively kas tool. - very good but a technology lock.

2.1.1.2. Gather build specification

2.1.2. Terminology

Developer build
  • Performed by a developer on a developers computer
  • Purpose is pure development, one build “small” target only
Verification build
  • Performed by a ci service on a headless computer
  • Fast feedback if the change/pull request might break master branch/trunk
Integration build
  • Performed by a ci service on a headless computer
  • Fast feedback if current state of integrated software still works
  • Provides latest integrated software in all variants
Integration release build
  • Performed by a ci service on a headless computer
  • Input is a pinned configuration
  • Provides reproducible and identifiable (by version) integrated software in all variants

2.2. Requirements

2.2.1. Use Cases

2.2.1.1. Developer bootstraps Yocto build environment

ronto bootstrap

Developer has to answer a couple of questions. Result is a created project directory plus a ronto.yml inside the project directory.

2.2.1.2. Developer builds something on his machine

Developer builds locally on his machine a one specific package and deploys it to his private development bare metal target.

$ ronto build --fetch --init --interactive
(yocto)> bitbake

2.2.1.3. Integrator builds locally

Integrator builds locally on his machine an image for two targets and deploy and tests to to integration targets.

ronto --env TARGETS=mytargets.yml build --fetch --init

2.2.1.4. CI server builds and publishes

Continuous Integration Server builds a set of images for a set of machines and publishes some images as well as all packages

ronto fetch
ronto init --cleanconf
ronto --env TARGETS=sources/recipe-repo/conf/integration-targets.yml build
ronto --env TARGETS=sources/recipe-repo/conf/integration-targets.yml publish

2.2.1.5. Release Engineer releases

A release engineer pins/releases a certain successful CI build. afterwards he/she build the pinned release.

# do some pinning stuff ronto –env MANIFEST=release_xyz.yml fetch ronto init –cleanbuild ronto –env TARGETS=sources/recipe-repo/conf/release-targets.yml build ronto –env TARGETS=sources/recipe-repo/conf/release-targets.yml publish

2.2.2. Requirements

  • This tool shall work with or without docker
  • This tool shall work with and without repo tool.

This tool is a convenient wrapper: Yocto builds with or without this tool shall work the same way.

2.2.2.1. Quality Attributes

  • The project config file is the central play that should operate with references to details that are rather bound to recipes and configuration repositories.
  • CLI and config file shall be self-explaining and intuitive
  • Different pront colors (interactive if on docker/bare metal
  • Different promt color if bitbake environment is active

2.3. Concept

Problems to be solved:

  • support of docker environments | bare metal environments
  • private repositories of sources (credential handling in docker)
  • handling site.conf
  • version pinning of integration releases
  • Consistent CI/Releasing commit messages and tags (established conventions)

2.3.1. Include of sub-rontofiles

Quality requirement: Maintainability

The build targets specification has to be located in the custom recipes repository. Only in that repository are machines and images known. Following the “strong coupling principle” the build targets specification must be maintained in that repository as well.

Note

Late loading must be implemented, since it might be possible that this specification is only or updated available after the fetching step

2.3.2. Cleanup flags

E.g. in rontofile:

build:
  flags:
    - cleanconf
    - cleanbuild
    - cleansstate

2.3.2.1. Solution:

Are part of init sub-command can be customized into *ronto*scripts.

2.3.3. Command line variables

Like reading from environment it is possibible to read from command line (as global parameter)

2.3.4. Compose of own Commands

Allow custom commands (including options + injections) like git config alias.xxx “blablub”

scripts:
  do-special:
    - '--env REPO=foo fetch'
    - '-f integrate.yml build'
    - '-f integrate.yml publish'

Pro:

  • more flexibility to handle e.g. process specifics of integration builds or release builds
  • support of customized developer workflows / development cycles

Con:

  • more complexity

Solution:

  • is implemented as proposed redesigned as run command and scripts section.

2.3.5. Multiple prioritized config files

  • Must be given at command line. Later on the command line implies higher priority.
  • Items at higher priority overwrite items with lower priority. (same mechanism like css)

Pro:

  • flexibility
  • homogeneous handling
  • little effort

Con:

  • Overwrite rules are not super intuitive
  • Concepts are not easily readable
  • (like salt: Problem solving if you only have a hammer, everything turns a nail)

Solution:

  • same thing is achived via extra “scripts” section, variable substituion, and include e.g. of target file.

2.4. Open Issues

2.4.1. Generate site.conf

Pro:
  • intelligent mapping between docker and none docker directories
  • no additional file to store
Con:
  • one more concept
### -- Not implemented yet - still subject to evaluation --
## Generate build/conf/site.conf from values
## to do
## either with semantics for distro, upstream, download, sstate_cache
## or from list of define strings
generate:
    download: "download"
    shared_state: "shared-state"
    distro: "{{ ams }}"

ronto --env FOO=bar fetch
ronto -eBAR=foo build

Pro:

  • compose of own commands is better supported
  • scripting is better since amount of site effects is minimized

Con:

  • little more effort