bpt CI Scripts Python API

Test Fixtures

The following test fixtures are defined:

Module: bpt_ci

Module: bpt_ci.bootstrap

class bpt_ci.bootstrap.BootstrapMode(value)

How should be bootstrap our prior BPT executable?

Build = 'build'

Build one from source

Download = 'download'

Downlaod one from GitHub

Lazy = 'lazy'

If the prior executable exists, skip, otherwise download

Skip = 'skip'

Skip bootstrapping. Assume it already exists.

bpt_ci.bootstrap.get_bootstrap_exe(mode: bpt_ci.bootstrap.BootstrapMode) → bpt_ci.bpt.BPTWrapper

Obtain a bpt executable for the given bootstrapping mode

bpt_ci.bootstrap.pin_exe(fpath: pathlib.Path) → Iterator[pathlib.Path]

Create a copy of the file at fpath at an unspecified location, and yield that path.

This is needed if the executable would overwrite itself.

Module: bpt_ci.bpt

class bpt_ci.bpt.BPTWrapper(path: pathlib.Path, *, crs_cache_dir: Optional[Union[pathlib.PurePath, str]] = None, default_cwd: Optional[Union[pathlib.PurePath, str]] = None)

Wraps a ‘bpt’ executable with some convenience APIs that invoke various ‘bpt’ subcommands.

property always_args: proc.CommandLine

Arguments that are always given to every bpt subcommand

build(*, root: Pathish, toolchain: Pathish | None = None, build_root: Pathish | None = None, jobs: int | None = None, tweaks_dir: Pathish | None = None, with_tests: bool = True, repos: Iterable[Pathish] = (), more_args: proc.CommandLine | None = None, timeout: float | None = None, cwd: Pathish | None = None) → None

Run ‘bpt build’ with the given arguments.

Parameters

  • root – The root project directory.

  • toolchain – The toolchain to use for the build.

  • build_root – The root directory where the output will be written.

  • jobs – The number of jobs to use. Default is CPU-count + 2

  • tweaks_dir – Set the tweaks-dir for the build

  • with_tests – Toggle building and executing of tests.

  • repos – Repositories to use during the build.

  • more_args – Additional command-line arguments.

  • timeout – Timeout for the build subprocess.

  • cwd – Working directory for the bpt subprocess

build_deps(args: proc.CommandLine, *, repos: Iterable[Pathish] = (), toolchain: Pathish | None = None) → None

run the bpt build-deps subcommand.

clean(*, build_dir: Optional[pathlib.Path] = None, crs_cache: bool = True) → None

Clean out prior executable output, including the CRS cache and the build results at ‘build_dir’, if given.

clone() → bpt_ci.bpt.BPTWrapper

Create a clone of this object

compile_file(paths: Iterable[Pathish], *, tweaks_dir: Pathish | None = None, toolchain: Pathish | None = None, root: Pathish, build_root: Pathish | None = None, more_args: proc.CommandLine = ()) → None

Run ‘bpt compile-file’ for the given paths.

crs_cache_dir

The directory in which the CRS cache data will be stored

property crs_cache_dir_arg: proc.CommandLine

The arguments for --crs-cache-dir

default_cwd

The directory in which bpt commands will execute unless otherwise specified

path

The path to the wrapped bpt executable

pkg_prefetch(*, repos: Iterable[Union[pathlib.PurePath, str]], pkgs: Iterable[str] = ()) → None

Execute the bpt pkg prefetch subcommand

pkg_solve(*, repos: Iterable[Union[pathlib.PurePath, str]], pkgs: Iterable[str]) → None

Execute the bpt pkg solve subcommand

run(args: proc.CommandLine, *, cwd: Pathish | None = None, timeout: float | None = None) → None

Execute the ‘bpt’ executable with the given arguments

Parameters

  • args – The command arguments to give to bpt.

  • cwd – The working directory of the subprocess.

  • timeout – A timeout for the subprocess’s execution.

bpt_ci.bpt.T

A generic unbounded type parameter

alias of TypeVar(‘T’)

Module: bpt_ci.docs

bpt_ci.docs.audit_docrefs_main()

Entrypoint of bpt-audit-docrefs. Finds documentation pages referred to in the source code, searching for spelling errors, and searches for any error pages that are not referenced anywhere in the source files.

bpt_ci.docs.doc_refs_in_code(code: str) → Iterable[str]

Find referenced documentation pages in the given C++ code

bpt_ci.docs.doc_refs_in_file(filepath: pathlib.Path) → set[str]

Find referenced documenation pages in the given C++ source file

bpt_ci.docs.nearest(given: str, cands: Iterable[str]) → str | None

Find the nearest candidate string to the given string

Parameters

  • given – An input string.

  • cands – A set of candidate strings to match.

Returns

The string in cands that is most similar to given, or None otherwise.

bpt_ci.docs.normalize_docname(fpath: pathlib.Path, root: pathlib.Path) → str

Normalize the name of a documentation page relative to a documentation root

Parameters

  • fpath – The absolute path of the documentation page file.

  • root – The absolute path of the documentation root directory.

bpt_ci.docs.scan_doc_references(src_root: pathlib.Path) → dict[pathlib.Path, set[str]]

Find referenced documentation pages in the given C++ source tree

Module: bpt_ci.msvs

class bpt_ci.msvs.Arguments(*args, **kwargs)

Module: bpt_ci.paths

bpt_ci.paths.BUILD_DIR = PosixPath('/home/colby/Documents/Programming/ddslim/_build')

The default build directory

bpt_ci.paths.CUR_BUILT_BPT = PosixPath('/home/colby/Documents/Programming/ddslim/_build/bpt')

The path to the main built bpt executable

bpt_ci.paths.EXE_SUFFIX = ''

The suffix of executable files on this system

bpt_ci.paths.PREBUILT_BPT = PosixPath('/home/colby/Documents/Programming/ddslim/_prebuilt/bpt')

The path to the prebuilt bpt executable

bpt_ci.paths.PREBUILT_DIR = PosixPath('/home/colby/Documents/Programming/ddslim/_prebuilt')

The directory were prebuild/bootstrapped results will go, and scratch space for the build

bpt_ci.paths.PROJECT_ROOT = PosixPath('/home/colby/Documents/Programming/ddslim')

The root directory of the bpt project

bpt_ci.paths.TESTS_DIR = PosixPath('/home/colby/Documents/Programming/ddslim/tests')

The <repo>/tests directory

bpt_ci.paths.TOOLS_DIR = PosixPath('/home/colby/Documents/Programming/ddslim/tools')

The <repo>/tools directory

bpt_ci.paths.TWEAKS_DIR = PosixPath('/home/colby/Documents/Programming/ddslim/conf')

The directory for tweak headers used by the build

bpt_ci.paths.find_exe(name: str) → Optional[pathlib.Path]

Find a file on the system by searching through the PATH environment variable.

bpt_ci.paths.new_tempdir() → Iterator[pathlib.Path]

Create and yield a new temporary directory, which will be destroyed on context-manager exit

Module: bpt_ci.proc

class bpt_ci.proc.ProcessResult(*args, **kwargs)

Module: bpt_ci.util

bpt_ci.util.Pathish

A path, string, or convertible-to-Path object

alias of Union[pathlib.PurePath, str]

Module: bpt_ci.toolchain

bpt_ci.toolchain.fixup_toolchain(json_file: Union[pathlib.PurePath, str]) → Iterator[pathlib.Path]

Augment the toolchain at the given path by adding ‘ccache’ or -fuse-ld=lld, if those tools are available on the system. Yields a new toolchain file based on ‘json_file’

bpt_ci.toolchain.get_default_audit_toolchain() → pathlib.Path

Get the default toolchain that should be used for dev and test based on the host platform.

bpt_ci.toolchain.get_default_test_toolchain() → pathlib.Path

Get the default toolchain that should be used by tests that need a toolchain to use for executing bpt.

bpt_ci.toolchain.get_default_toolchain() → pathlib.Path

Get the default toolchain that should be used to generate the release executable based on the host platform.

Module: bpt_ci.testing

class bpt_ci.testing.CRSRepo(path: pathlib.Path, bpt: bpt_ci.bpt.BPTWrapper)

A CRS repository directory

class bpt_ci.testing.DirRenderer(cache_root: Union[pathlib.PurePath, str], tmp_root: Union[pathlib.PurePath, str])

Utility for generating filesystem structures and restoring them from a cache.

Parameters

  • cache_root – The root directory where rendered content will be stored.

  • tmp_root – Root directory for temporary files.

get_or_prepare(key: str) → ContextManager[bpt_ci.testing.fs.GetResult]

Open a context manager to a directory that may or may not already be cached.

If GetResult.ready_path is not None, the directory is already present in the cache.

class bpt_ci.testing.Project(dirpath: pathlib.Path, bpt: bpt_ci.bpt.BPTWrapper)

Utilities to access a project being used as a test.

property bpt_yaml: bpt_ci.testing.fixtures.ProjectYAML

Get/set the content of the bpt.yaml file for the project.

build(*, toolchain: Optional[Union[pathlib.PurePath, str]] = None, fixup_toolchain: bool = True, jobs: Optional[int] = None, timeout: Optional[float] = None, tweaks_dir: Optional[pathlib.Path] = None, with_tests: bool = True, repos: Sequence[Union[pathlib.PurePath, str]] = (), log_level: Literal['info', 'debug', 'trace'] = 'trace', cwd: Optional[Union[pathlib.PurePath, str]] = None) → None

Execute ‘bpt build’ on the project

property project_dir_arg: str

Argument for –project

write(path: Union[pathlib.PurePath, str], content: str) → pathlib.Path

Write the given content to path. If path is relative, it will be resolved relative to the root directory of this project.

class bpt_ci.testing.ProjectOpener(bpt: bpt_ci.bpt.BPTWrapper, request: _pytest.fixtures.FixtureRequest, worker: str, tmp_path_factory: _pytest.tmpdir.TempPathFactory, dir_renderer: bpt_ci.testing.fs.DirRenderer)

A test fixture that opens project directories for testing

open(dirpath: Union[pathlib.PurePath, str]) → bpt_ci.testing.fixtures.Project

Open a new project testing fixture from the given project directory.

Parameters

dirpath – The directory that contains the project to use.

Clones the given directory and then opens a project within that clone. The clone directory will be destroyed when the test fixture is torn down.

property test_dir: pathlib.Path

The directory that contains the test that requested this opener

property test_name: str

The name of the test that requested this opener

class bpt_ci.testing.ProjectYAML(_typename, _fields=None, /, **kwargs)

Module: bpt_ci.testing.error

Test utilities for error checking

bpt_ci.testing.error.expect_error_marker(expect: str) → ContextManager[None]

A context-manager function that should wrap a scope that causes an error from bpt.

Parameters

expect – A string description of the expected marker

The wrapped scope should raise subprocess.CalledProcessError.

After handling the exception, asserts that the subprocess wrote an error marker containing the string given in expect.

bpt_ci.testing.error.expect_error_marker_pred(pred: Callable[[str], bool], expected: str) → Iterator[None]

A context-manager function that should wrap a scope that causes an error from bpt.

Parameters

  • pred – A predicate which checks if the marker passes

  • expected – A string description of the expected marker

The wrapped scope should raise subprocess.CalledProcessError.

After handling the exception, asserts that the subprocess wrote an error marker matching pred.

bpt_ci.testing.error.expect_error_marker_re(expect: Union[str, Pattern[str]]) → ContextManager[None]

A context-manager function that should wrap a scope that causes an error from bpt.

Parameters

expect – A regular expression that the expected marker should match

The wrapped scope should raise subprocess.CalledProcessError.

After handling the exception, asserts that the subprocess wrote an error marker matching expect.

Module: bpt_ci.testing.fixtures

Test fixtures used by BPT in pytest

class bpt_ci.testing.fixtures.Library(name: str, dirpath: pathlib.Path)

Utilities to access a library under libs/ for a Project.

write(path: Union[pathlib.PurePath, str], content: str) → pathlib.Path

Write the given content to path. If path is relative, it will be resolved relative to the root directory of this library.

class bpt_ci.testing.fixtures.Project(dirpath: pathlib.Path, bpt: bpt_ci.bpt.BPTWrapper)

Utilities to access a project being used as a test.

property bpt_yaml: bpt_ci.testing.fixtures.ProjectYAML

Get/set the content of the bpt.yaml file for the project.

build(*, toolchain: Optional[Union[pathlib.PurePath, str]] = None, fixup_toolchain: bool = True, jobs: Optional[int] = None, timeout: Optional[float] = None, tweaks_dir: Optional[pathlib.Path] = None, with_tests: bool = True, repos: Sequence[Union[pathlib.PurePath, str]] = (), log_level: Literal['info', 'debug', 'trace'] = 'trace', cwd: Optional[Union[pathlib.PurePath, str]] = None) → None

Execute ‘bpt build’ on the project

property project_dir_arg: str

Argument for –project

write(path: Union[pathlib.PurePath, str], content: str) → pathlib.Path

Write the given content to path. If path is relative, it will be resolved relative to the root directory of this project.

class bpt_ci.testing.fixtures.ProjectOpener(bpt: bpt_ci.bpt.BPTWrapper, request: _pytest.fixtures.FixtureRequest, worker: str, tmp_path_factory: _pytest.tmpdir.TempPathFactory, dir_renderer: bpt_ci.testing.fs.DirRenderer)

A test fixture that opens project directories for testing

open(dirpath: Union[pathlib.PurePath, str]) → bpt_ci.testing.fixtures.Project

Open a new project testing fixture from the given project directory.

Parameters

dirpath – The directory that contains the project to use.

Clones the given directory and then opens a project within that clone. The clone directory will be destroyed when the test fixture is torn down.

property test_dir: pathlib.Path

The directory that contains the test that requested this opener

property test_name: str

The name of the test that requested this opener

class bpt_ci.testing.fixtures.ProjectYAML(_typename, _fields=None, /, **kwargs)
bpt_ci.testing.fixtures.bpt(bpt_exe: pathlib.Path) → bpt_ci.bpt.BPTWrapper

A BPTWrapper around the bpt executable under test

bpt_ci.testing.fixtures.bpt_exe(pytestconfig: _pytest.config.Config) → pathlib.Path

A pathlib.Path pointing to the BPT executable under test

bpt_ci.testing.fixtures.fixture_fspath(req: _pytest.fixtures.FixtureRequest) → pathlib.Path

Return the fspath associated with the given test fixture request.

Note

This function is only a workaround for a missing type annotation on FixtureRequest

bpt_ci.testing.fixtures.project_opener(request: _pytest.fixtures.FixtureRequest, worker_id: str, bpt: bpt_ci.bpt.BPTWrapper, tmp_path_factory: _pytest.tmpdir.TempPathFactory, dir_renderer: bpt_ci.testing.fs.DirRenderer) → bpt_ci.testing.fixtures.ProjectOpener

A fixture factory that can open directories as Project objects for building and testing. Duplicates the project directory into a temporary location so that the original test directory remains unchanged.

bpt_ci.testing.fixtures.test_parent_dir(request: _pytest.fixtures.FixtureRequest) → pathlib.Path

pathlib.Path fixture pointing to the parent directory of the file containing the test that is requesting the current fixture

bpt_ci.testing.fixtures.tmp_git_repo_factory(tmp_path_factory: _pytest.tmpdir.TempPathFactory, pytestconfig: _pytest.config.Config) → Callable[[Union[pathlib.PurePath, str]], pathlib.Path]

A temporary directory pathlib.Path object in which a git repo will be initialized

bpt_ci.testing.fixtures.tmp_project(request: _pytest.fixtures.FixtureRequest, worker_id: str, project_opener: bpt_ci.testing.fixtures.ProjectOpener, tmp_path_factory: _pytest.tmpdir.TempPathFactory) → bpt_ci.testing.fixtures.Project

A fixture that generates an empty temporary project directory that will be thrown away when the test completes.

Module: bpt_ci.testing.fs

class bpt_ci.testing.fs.DirRenderer(cache_root: Union[pathlib.PurePath, str], tmp_root: Union[pathlib.PurePath, str])

Utility for generating filesystem structures and restoring them from a cache.

Parameters

  • cache_root – The root directory where rendered content will be stored.

  • tmp_root – Root directory for temporary files.

get_or_prepare(key: str) → ContextManager[bpt_ci.testing.fs.GetResult]

Open a context manager to a directory that may or may not already be cached.

If GetResult.ready_path is not None, the directory is already present in the cache.

class bpt_ci.testing.fs.GetResult(ready_path, prep_path, final_dest)
final_dest: pathlib.Path

Alias for field number 2

prep_path: Optional[pathlib.Path]

Alias for field number 1

ready_path: Optional[pathlib.Path]

If not None, the path to the already-generated directory. Returned by DirRenderer.get_or_prepare

Module: bpt_ci.testing.http

class bpt_ci.testing.http.DirectoryServingHTTPRequestHandler(*args: Any, **kwargs: Any)

A simple HTTP request handler that simply serves files from a directory given to the constructor.

translate_path(path: str) → str

Translate a /-separated PATH to the local filename syntax.

Components that mean special things to the local file system (e.g. drive or directory names) are ignored. (XXX They should probably be diagnosed.)

class bpt_ci.testing.http.ServerInfo(base_url: str, root: pathlib.Path)

Information about an HTTP server fixture

base_url: str

Alias for field number 0

root: pathlib.Path

Alias for field number 1

bpt_ci.testing.http.http_server_factory(request: _pytest.fixtures.FixtureRequest) → Callable[[pathlib.Path], bpt_ci.testing.http.ServerInfo]

Spawn an HTTP server that serves the content of a directory.

bpt_ci.testing.http.run_http_server(dirpath: pathlib.Path, port: int) → Iterator[bpt_ci.testing.http.ServerInfo]

Context manager that spawns an HTTP server that serves thegiven directory on the given TCP port.

Module: bpt_ci.testing.repo

Test fixtures and utilities for creating and using CRS repositories

class bpt_ci.testing.repo.CRSRepo(path: pathlib.Path, bpt: bpt_ci.bpt.BPTWrapper)

A CRS repository directory

class bpt_ci.testing.repo.CRSRepoServer(repo: bpt_ci.testing.repo.CRSRepo, server: bpt_ci.testing.http.ServerInfo)

An HTTP-server CRS repository

repo: bpt_ci.testing.repo.CRSRepo

Alias for field number 0

server: bpt_ci.testing.http.ServerInfo

Alias for field number 1

bpt_ci.testing.repo.http_crs_repo(tmp_crs_repo: bpt_ci.testing.repo.CRSRepo, http_server_factory: Callable[[pathlib.Path], bpt_ci.testing.http.ServerInfo]) → bpt_ci.testing.repo.CRSRepoServer

Generate a temporary HTTP server serving a temporary CRS repository

External Types to Know

These types are defined externally and arg used extensively throughout the CI scripts.

class _pytest.fixtures.FixtureRequest

See also

pytest.FixtureRequest

class _pytest.tmpdir.TempPathFactory

See also

pytest.TempPathFactory

class _pytest.config.Config

See also

pytest.Config

class bpt_ci.testing.repo.Pathish

A type alias for objects that can be treated as paths. Includes any subclass of PurePath. str, and types providing an __fspath__ method.

class dagon.proc.CommandLine

A sequence of command-line-argument compatible types, including Pathish, int, float, and recursive CommandLine arrays.