siibra.retrieval.repositories

Connect to repositories to browse and pull files within.

Module Contents

Classes

EbrainsHdgConnector	Download sensitive files from EBRAINS using
EbrainsPublicDatasetConnector	Access files from public EBRAINS datasets via the Knowledge Graph v3 API.
EbrainsPublicDatasetConnectorMinds	Access files from public EBRAINS datasets via the Knowledge Graph v3 API.
GithubConnector	Base class for repository connectors.
GitlabConnector	Base class for repository connectors.
LocalFileRepository	Base class for repository connectors.
OwncloudConnector	Base class for repository connectors.
RepositoryConnector	Base class for repository connectors.
ZipfileConnector	Base class for repository connectors.

class siibra.retrieval.repositories.EbrainsHdgConnector(dataset_id)

Download sensitive files from EBRAINS using the Human Data Gateway (HDG) via the data proxy API. Service documentation can be found here https://data-proxy.ebrains.eu/api/docs

api_version = 'v1'

Base URL for the Dataset Endpoint of the Data-Proxy API https://data-proxy.ebrains.eu/api/docs#/datasets

Supported functions by the endpoint:

• POST: Request access to the dataset.

  This is required for the other functions.

• GET: Return list of all available objects in the dataset

base_url

Limit of returned objects Default value on API side is 50 objects

maxentries = 1000

get_loader(filename, folder='', decode_func=None)

Get a lazy loader for a file, for executing the query only once loader.data is accessed.

search_files(folder='', suffix=None, recursive=False)

Get the files within the repository.

Parameters:

• folder (str) – folder or folders in the form ‘path/to/file’

• suffix (str) –

• recursive (bool, default: False) – If True, searches files in all subfolders

Returns:

List of file names.

Return type:

List[str]

class siibra.retrieval.repositories.EbrainsPublicDatasetConnector(dataset_id: str = None, version_id: str = None, title: str = None, in_progress=False)

Access files from public EBRAINS datasets via the Knowledge Graph v3 API.

property authors

property citation

property description

property name

QUERY_ID = 'bebbe365-a0d6-41ea-9ff8-2554c15f70b7'

base_url = 'https://core.kg.ebrains.eu/v3-beta/queries/'

maxentries = 1000

get_loader(filename, folder='', decode_func=None)

Get a lazy loader for a file, for executing the query only once loader.data is accessed.

search_files(folder='', suffix=None, recursive=False)

Get the files within the repository.

Parameters:

• folder (str) – folder or folders in the form ‘path/to/file’

• suffix (str) –

• recursive (bool, default: False) – If True, searches files in all subfolders

Returns:

List of file names.

Return type:

List[str]

class siibra.retrieval.repositories.EbrainsPublicDatasetConnectorMinds(dataset_id=None, title=None, in_progress=False)

Access files from public EBRAINS datasets via the Knowledge Graph v3 API.

QUERY_ID = 'siibra-minds-dataset-v1'

base_url = 'https://kg.humanbrainproject.eu/query/minds/core/dataset/v1.0.0'

maxentries = 1000

get_loader(filename, folder='', decode_func=None)

Get a lazy loader for a file, for executing the query only once loader.data is accessed.

search_files(folder='', suffix=None, recursive=False)

Get the files within the repository.

Parameters:

• folder (str) – folder or folders in the form ‘path/to/file’

• suffix (str) –

• recursive (bool, default: False) – If True, searches files in all subfolders

Returns:

List of file names.

Return type:

List[str]

class siibra.retrieval.repositories.GithubConnector(owner: str, repo: str, reftag: str, skip_branchtest=False, archive_mode=False)

Base class for repository connectors.

get_loader(filename, folder='', decode_func=None)

Get a lazy loader for a file, for executing the query only once loader.data is accessed.

search_files(folder='', suffix='', recursive=False) → List[str]

Get the files within the repository.

Parameters:

• folder (str) – folder or folders in the form ‘path/to/file’

• suffix (str) –

• recursive (bool, default: False) – If True, searches files in all subfolders

Returns:

List of file names.

Return type:

List[str]

class siibra.retrieval.repositories.GitlabConnector(server: str, project: int, reftag: str, skip_branchtest=False, *, archive_mode=False)

Base class for repository connectors.

property branches

property want_commit

__eq__(other)

Return self==value.

__str__()

Return str(self).

get_loader(filename, folder='', decode_func=None)

Get a lazy loader for a file, for executing the query only once loader.data is accessed.

search_files(folder='', suffix=None, recursive=False)

Get the files within the repository.

Parameters:

• folder (str) – folder or folders in the form ‘path/to/file’

• suffix (str) –

• recursive (bool, default: False) – If True, searches files in all subfolders

Returns:

List of file names.

Return type:

List[str]

class siibra.retrieval.repositories.LocalFileRepository(folder: str)

Base class for repository connectors.

class FileLoader(file_url, decode_func)

Just a loads a local file, but mimics the behaviour of cached http requests used in other connectors.

property data

__eq__(other)

Return self==value.

__str__()

Return str(self).

get_loader(filename, folder='', decode_func=None)

Get a lazy loader for a file, for loading data only once loader.data is accessed.

search_files(folder='', suffix=None, recursive=False)

Get the files within the repository.

Parameters:

• folder (str) – folder or folders in the form ‘path/to/file’

• suffix (str) –

• recursive (bool, default: False) – If True, searches files in all subfolders

Returns:

List of file names.

Return type:

List[str]

class siibra.retrieval.repositories.OwncloudConnector(server: str, share: int)

Base class for repository connectors.

abstract search_files(folder='', suffix=None, recursive=False)

Get the files within the repository.

Parameters:

• folder (str) – folder or folders in the form ‘path/to/file’

• suffix (str) –

• recursive (bool, default: False) – If True, searches files in all subfolders

Returns:

List of file names.

Return type:

List[str]

class siibra.retrieval.repositories.RepositoryConnector(base_url)

Base class for repository connectors.

__eq__(other)

Return self==value.

get(filename, folder='', decode_func=None)

Get a file right away.

get_loader(filename, folder='', decode_func=None)

Get a lazy loader for a file, for executing the query only once loader.data is accessed.

get_loaders(folder='', suffix=None, progress=None, recursive=False, decode_func=None)

Returns an iterator with lazy loaders for the files in a given folder. In each iteration, a tuple (filename,file content) is returned.

abstract search_files(folder: str, suffix: str, recursive: bool = False) → List[str]

Get the files within the repository.

Parameters:

• folder (str) – folder or folders in the form ‘path/to/file’

• suffix (str) –

• recursive (bool, default: False) – If True, searches files in all subfolders

Returns:

List of file names.

Return type:

List[str]

class siibra.retrieval.repositories.ZipfileConnector(url: str)

Base class for repository connectors.

class ZipFileLoader(zipfile, filename, decode_func)

Loads a file from the zip archive, but mimics the behaviour of cached http requests used in other connectors.

property cached

property data

property zipfile

__eq__(other)

Return self==value.

__str__()

Return str(self).

clear_cache()

get_loader(filename, folder='', decode_func=None)

Get a lazy loader for a file, for loading data only once loader.data is accessed.

search_files(folder='', suffix='', recursive=False)

Get the files within the repository.

Parameters:

• folder (str) – folder or folders in the form ‘path/to/file’

• suffix (str) –

• recursive (bool, default: False) – If True, searches files in all subfolders

Returns:

List of file names.

Return type:

List[str]