siibra.retrieval

File digestion and caching.

Submodules

Attributes

CACHE

Exceptions

NoSiibraConfigMirrorsAvailableException

Common base class for all non-exit exceptions.

SiibraHttpRequestError

Common base class for all non-exit exceptions.

TagNotFoundException

Common base class for all non-exit exceptions.

Classes

EbrainsHdgConnector

Download sensitive files from EBRAINS using

EbrainsPublicDatasetConnector

Access files from public EBRAINS datasets via the Knowledge Graph v3 API.

EbrainsRequest

Implements lazy loading of HTTP Knowledge graph queries.

GitlabConnector

Base class for repository connectors.

HttpRequest

LocalFileRepository

Base class for repository connectors.

OwncloudConnector

Base class for repository connectors.

ZipfileConnector

Base class for repository connectors.

ZipfileRequest

Package Contents

exception siibra.retrieval.NoSiibraConfigMirrorsAvailableException
Inheritance diagram of siibra.retrieval.NoSiibraConfigMirrorsAvailableException

Common base class for all non-exit exceptions.

exception siibra.retrieval.SiibraHttpRequestError(url: str, status_code: int, msg='Cannot execute http request.')
Inheritance diagram of siibra.retrieval.SiibraHttpRequestError

Common base class for all non-exit exceptions.

__str__()

Return str(self).

msg = 'Cannot execute http request.'
status_code
url
exception siibra.retrieval.TagNotFoundException
Inheritance diagram of siibra.retrieval.TagNotFoundException

Common base class for all non-exit exceptions.

class siibra.retrieval.EbrainsHdgConnector(dataset_id)
Inheritance diagram of siibra.retrieval.EbrainsHdgConnector

Download sensitive files from EBRAINS using the Human Data Gateway (HDG) via the data proxy API. Service documentation can be found here https://data-proxy.ebrains.eu/api/docs

get_loader(filename, folder='', decode_func=None)

Get a lazy loader for a file, for executing the query only once loader.data is accessed.

search_files(folder='', suffix=None, recursive=False)

Get the files within the repository.

Parameters:
  • folder (str) – folder or folders in the form ‘path/to/file’

  • suffix (str) –

  • recursive (bool, default: False) – If True, searches files in all subfolders

Returns:

List of file names.

Return type:

List[str]

api_version = 'v1'

Base URL for the Dataset Endpoint of the Data-Proxy API https://data-proxy.ebrains.eu/api/docs#/datasets

Supported functions by the endpoint:

  • POST: Request access to the dataset.

    This is required for the other functions.

  • GET: Return list of all available objects in the dataset

base_url

Limit of returned objects Default value on API side is 50 objects

dataset_id
maxentries = 1000
class siibra.retrieval.EbrainsPublicDatasetConnector(dataset_id: str = None, version_id: str = None, title: str = None, in_progress=False)
Inheritance diagram of siibra.retrieval.EbrainsPublicDatasetConnector

Access files from public EBRAINS datasets via the Knowledge Graph v3 API.

get_loader(filename, folder='', decode_func=None)

Get a lazy loader for a file, for executing the query only once loader.data is accessed.

search_files(folder='', suffix=None, recursive=False)

Get the files within the repository.

Parameters:
  • folder (str) – folder or folders in the form ‘path/to/file’

  • suffix (str) –

  • recursive (bool, default: False) – If True, searches files in all subfolders

Returns:

List of file names.

Return type:

List[str]

QUERY_ID = 'bebbe365-a0d6-41ea-9ff8-2554c15f70b7'
property authors
base_url = 'https://core.kg.ebrains.eu/v3-beta/queries/'
property citation
dataset_id = None
property description
id
maxentries = 1000
property name
use_version = None
versions
class siibra.retrieval.EbrainsRequest(url, decoder=None, params={}, msg_if_not_cached=None, post=False)
Inheritance diagram of siibra.retrieval.EbrainsRequest

Implements lazy loading of HTTP Knowledge graph queries.

classmethod device_flow(**kwargs)
classmethod fetch_token(**kwargs)

Fetch an EBRAINS token using commandline-supplied username/password using the data proxy endpoint.

Details on how to access EBRAINS are here.

get()

Evaluate KG Token is evaluated only on execution of the request.

classmethod init_oidc()
classmethod set_token(token)
property auth_headers
keycloak_endpoint = 'https://iam.ebrains.eu/auth/realms/hbp/protocol/openid-connect/token'
property kg_token
params
class siibra.retrieval.GitlabConnector(server: str, project: int, reftag: str, skip_branchtest=False, *, archive_mode=False)
Inheritance diagram of siibra.retrieval.GitlabConnector

Base class for repository connectors.

__eq__(other)
__str__()
get_loader(filename, folder='', decode_func=None)

Get a lazy loader for a file, for executing the query only once loader.data is accessed.

search_files(folder='', suffix=None, recursive=False)

Get the files within the repository.

Parameters:
  • folder (str) – folder or folders in the form ‘path/to/file’

  • suffix (str) –

  • recursive (bool, default: False) – If True, searches files in all subfolders

Returns:

List of file names.

Return type:

List[str]

archive_mode = False
property branches
reftag
property want_commit
class siibra.retrieval.HttpRequest(url: str, func: Callable = None, msg_if_not_cached: str = None, refresh=False, post=False, **kwargs)
get()
property cached
cachefile
property data
kwargs
msg_if_not_cached = None
post = False
refresh = False
url
class siibra.retrieval.LocalFileRepository(folder: str)
Inheritance diagram of siibra.retrieval.LocalFileRepository

Base class for repository connectors.

__eq__(other: LocalFileRepository)
__str__()
get_loader(filename, folder='', decode_func=None)

Get a lazy loader for a file, for loading data only once loader.data is accessed.

search_files(folder='', suffix=None, recursive=False)

Get the files within the repository.

Parameters:
  • folder (str) – folder or folders in the form ‘path/to/file’

  • suffix (str) –

  • recursive (bool, default: False) – If True, searches files in all subfolders

Returns:

List of file names.

Return type:

List[str]

class siibra.retrieval.OwncloudConnector(server: str, share: int)
Inheritance diagram of siibra.retrieval.OwncloudConnector

Base class for repository connectors.

abstract search_files(folder='', suffix=None, recursive=False)

Get the files within the repository.

Parameters:
  • folder (str) – folder or folders in the form ‘path/to/file’

  • suffix (str) –

  • recursive (bool, default: False) – If True, searches files in all subfolders

Returns:

List of file names.

Return type:

List[str]

class siibra.retrieval.ZipfileConnector(url: str)
Inheritance diagram of siibra.retrieval.ZipfileConnector

Base class for repository connectors.

class ZipFileLoader(zipfile, filename, decode_func)

Loads a file from the zip archive, but mimics the behaviour of cached http requests used in other connectors.

property cached
cachefile
property data
filename
func
zipfile
__eq__(other)
__str__()
clear_cache()
get_loader(filename, folder='', decode_func=None)

Get a lazy loader for a file, for loading data only once loader.data is accessed.

search_files(folder='', suffix='', recursive=False)

Get the files within the repository.

Parameters:
  • folder (str) – folder or folders in the form ‘path/to/file’

  • suffix (str) –

  • recursive (bool, default: False) – If True, searches files in all subfolders

Returns:

List of file names.

Return type:

List[str]

url
property zipfile
class siibra.retrieval.ZipfileRequest(url, filename, func=None, refresh=False)
Inheritance diagram of siibra.retrieval.ZipfileRequest
get()
filename
siibra.retrieval.CACHE = None