lor package

Submodules

lor.cmdline module

Top-level LoR CLI launcher

lor.cmdline.get_default_out_of_workspace_subcommands()

Returns a dict of default out-of-workspace subcommands as <name: CliCommand>s

Returns:A dict of <name: CliCommand>
lor.cmdline.get_default_workspace_subcommands()

Returns a dict of default in-workspace subcommands as <name: CliCommand>s

Return A dict of <name:
 CliCommand>
lor.cmdline.launch(out_of_workspace_subcommands=None, workspace_subcommands=None)

Launch LoR top-level CLI

Parameters:
  • out_of_workspace_subcommands – A dict of <name: CliCommand>s to use when outside a workspace
  • workspace_subcommands – A dict of <name: CliCommand>s to use when inside a workspace

lor.pathalias module

lor.properties module

lor.test module

Testing support

This module contains helper functions that downstream workspaces might want to use for testing purposes.

class lor.test.ChangeWorkspace(ws_path)

Bases: object

Set the global workspace temporarily.

class lor.test.TemporaryEnv

Bases: object

Enable the application’s environment (env vars, pwd) to be temporarily mutated, resetting it afterwards.

class lor.test.TemporaryHdfsDir

Bases: object

Create a temporary directory on HDFS

The directory’s path on HDFS is returned. After the block exists, the dir is deleted. Useful for testing tasks on-cluster without filling the cluster’s HDFS filesystem with test outputs.

class lor.test.TemporaryWorkspace

Bases: object

Create a temporary workspace

Creates a workspace and sets it as the global workspace temporarily.

lor.test.assert_task_fails(luigi_task)

Assert that luigi_task cannot be executed on a local scheduler.

An AssertionError is raised if the task executed successfully.

Parameters:luigi_task – A fully-instantiated Luigi task
lor.test.assert_task_passes(luigi_task)

Assert that luigi_task can be executed on a local scheduler.

An AssertionError is raised if the task cannot be executed

Parameters:luigi_task – A fully-instantiated Luigi task

lor.workspace module

Workspace support.

A workspace is a structured, but standard, python project, which enables downstream projects to use a “convention over configuration” design. Definitions in this module make it easier to work with that structure (e.g. creating it, establishing whether a directory is a workspace, resolving paths in the workspace).

This module also provides global access to the current workspace, which can be established automatically (from the current working dir or an env var) or set explicitly (e.g. by LoR’s CLI commands). This enables downstream code to perform working-directory-independent pathing.

lor.workspace.get_package_name(ws_path)

Returns the name of a workspace’s main python package.

Returns:

The workspace’s package name
Raises:
  • ValueError – if ws_path is not a workspace
  • Exception – If the package name cannot be found from information in the workspace
lor.workspace.get_path(cwd=None)

Returns the current workspace’s path, or None if the workspace has not been set and cannot be located automatically.

Returns:The current workspace’s path as a string, or None if the current workspace cannot be established.
lor.workspace.is_workspace(path)

Return True if path is a workspace; otherwise, return False.

Throws if the path does not exist or is a file.

Parameters:

path – The path to check
Returns:

True if path is a workspace. False if path is not a workspace.
Raises:
  • FileNotFoundError – if path does not exist
  • NotADirectoryError – if path is not a directory
lor.workspace.run_install_script(ws_path)

Run a workspace’s install script.

Parameters:

ws_path – Path to the workspace
Returns:

True if the install script ran successfully; otherwise, False
Raises:
  • FileNotFoundError – if workspace_path does not exist
  • NotADirectoryError – if workspace_path is not a directory
  • FileNotFoundError – if the install script cannot be found
lor.workspace.try_locate(cwd=None)

Returns a path to the workspace, if it can be found: otherwise, returns None.

This function looks for a workspace by:

  • Checking if LOR_HOME environment variable is set and trying that
  • Checking if cwd is a workspace

lor.workspace_global module

Module contents