Alpha
Python SDK

SeeMe.ai Python SDK #

The Python SDK is a convenient wrapper to communicate with the SeeMe.ai API.

This document provides a detailed overview of all the methods and their parameters.

Installation #

Install the SDK from the command line:

$ pip install --upgrade seeme

or in your notebook:

!pip install --upgrade seeme

Verfiy the version you have installed:

import seeme
seeme.__version__
'0.13.8'

This document describes the 0.13.7 SDK

Register/Login #

Create a client #

Create a client to interact with the SeeMe.ai API, allowing you to manage models, datasets, predictions and training requests.

from seeme import Client
client = Client()

parameters:

  • username (optional) : the username for the account you want to use;
  • apikey (optional): the API key for the username you want user;
  • backend (default provided): the backend the client communicates with, default “https://api.seeme.ai/api/v1.

Note:

Although username and apikey are optional, they need to be used together in order to be authenticated.

If you have a username and apikey, you can create a client that is ready to go:

my_username = ""
my_apikey = ""

client = Client(username=my_username, apikey=my_apikey)

Advanced: create a client that talks to a custom backend:

alternative_url = ""

client = Client(backend=alternative_url)

# or with auth
client = Client(username=my_username, apikey=my_apikey, backend=alternative_url)

Register #

Register a new user:

my_username =    # example: "my_username"
my_email =       # example: "jan.vandepoel@seeme.ai"
my_password =    # example: "supersecurepassword"
my_firstname =   # example: "Jan"
my_name =        # example: "Van de Poel"
client.register(
    username=my_username, 
    email=my_email, 
    password=my_password, 
    firstname=my_firstname, 
    name=my_name
)

Log in #

username = ""
password = ""

client.login(username, password)

Log out #

client.logout()

Models #

Get all models #

Get a list of all models you have access to.

models = client.get_models()

These models are divided in three groups:

  • public: models that are available to everyone;
  • owned: models you created;
  • shared: models that are shared privately with you.

Your own models

A list of the models you created:

own_models = [ model for model in models if model["user_id"] == client.user_id]

Public models

Public models are provided by SeeMe.ai or other users.

public_models = [model for model in models if model["public"]]

Shared models

A list of models that others have shared with you in private.

shared_with_me = [model for model in models if model["shared_with_me"]]

Create model #

my_model = {
    "name": "Cats and dogs",
    "description": "Recognize cats and dogs in pictures."
}

my_model = client.create_model(my_model)

Every model has the following properties:

{
    'id': '22791060-71f5-4e71-935e-28b52ef4c047',
     'created_at': '2021-10-28T08:49:06.698391719Z',
     'updated_at': '2021-10-28T08:49:06.698391719Z',
     'name': 'Cats and dogs',
     'description': 'Recognize cats and dogs in pictures.',
     'active_version_id': 'fc92b1fe-2ee0-4d99-9a09-0a37d0c68ea1',
     'classes': '',
     'accuracy': 0,
     'user_id': 'd7159432-f218-44ac-aebe-e5d661d62862',
     'can_inference': False,
     'kind': '',
     'has_logo': False,
     'logo': '',
     'public': False,
     'config': '',
     'application_id': '',
     'has_ml_model': False,
     'has_onnx_model': False,
     'has_tflite_model': False,
     'has_labels_file': False,
     'shared_with_me': False,
     'auto_convert': True,
     'privacy_enabled': False
 }

Properties in more detail:

PropertyDescription
idUnique id for the model
created_atThe creation date
updated_atLast updated date
nameThe model name
descriptionThe model description
accuracyThe model accuracy
user_idThe user id of the model creator
can_inferenceFlag indicating whether the model can make predictions or not
kindType of AI model, possible values: “image_classification”, “object_detection”, “text_classification”, “structured”
has_logoFlag indicating whether the model has a logo or not
logoName and extension of the logo file (mostly for internal purpose)
publicFlag indicating whether the model is public or not
configAdditional config stored in a JSON string
active_version_idThe id of the current model version (see versions below)
application_idThe application ID (see applications below)
has_ml_modelFlag indicating whether the model has a Core ML model
has_onnx_modelFlag indicating whether the model has an ONNX model
has_tflite_modelFlag indicating whether the model has a Tensorflow Lite model
has_labels_fileFlag indicating whether a file will all the labels (classes) is available
shared_with_meFlag indicating whether the model has been shared with you
auto_convertFlag indicating whether the model will be automatically converted to the supported model formats (see applications). Default value: True.
privacy_enabledFlag indicating whether privacy is enabled. If set to ‘True’, no inputs (images, text files, …) will be stored on the server, or the mobile/edge device. Default value: False.
# deprecated
# just calls `create_model` underneath

my_model = client.create_full_model(my_model)

Get model #

Use the model id to get all the metadata of the model:

client.get_model(my_model["id"])
ParametersDescription
model_idUnique id for the model

Update model #

Update any property of the model:

my_model["description"] = "Updated for demo purposes"

client.update_model(my_model)
ParametersDescription
modelThe entire model object

Delete model #

Delete a model using its id.

client.delete_model(my_model["id"])
ParametersDescription
modelThe entire model object

Upload model file #

You can upload the model file by calling the upload_model:

my_model = client.upload_model(my_model["id"], folder="directory/to/model", filename="cats_and_dogs.pkl")
ParametersDescription
model_idUnique id for the model
folderName of the folder that contains the model file (without trailing ‘/'), default value “data”
filenameName of the file to be uploaded, default value “export.pkl”

This returns an updated my_model, where if successful the can_inference should now be set to True.

If auto_convert is enabled (default value: True) and depending on the used application_id (see Applications below), the model will have updated values for has_ml_model, has_tflite_model, has_onnx_model, has_labels_file.

Download model file(s) #

Download the file(s) associated with the current model. The method is called download_active_model, because it will download the currently active (in production) model files. (See Versions for more downloads)

client.download_active_model(my_model, asset_type="pkl")
ParametersDescription
modelThe entire model object
asset_typeThe model type you want to download. Default: pkl; Possible values: mlmodel, tflite, onnx, names, labels

If the asset_type exists, the model file will be downloaded to my_model["active_model_id"].{asset_type}. One exception, the labels file will receive a .txt extension.

my_model = client.upload_logo(my_model["id"], folder="directory/to/logo", filename="logo_filename.jpg")
ParametersDescription
model_idUnique id for the model
folderName of the folder that contains the logo file (without trailing ‘/'), default value “data”
filenameName of the file to be uploaded, default value “logo.jpg”. Supported formats: jpg, jpeg, png.
client.get_logo(my_model)
ParametersDescription
modelThe entire model object

Make a prediction #

result = client.predict(model_id, item, input_type="image_classification")
ParametersDescription
model_idUnique id for the model
itemThe item you wish to make a prediction on:For “image_classification” and “object_detection” specify the full file location (including directory).For “text_classification” pass in the string you would like to predict.For “structured” pass in a JSON dump of the JSON object you want to use for your prediction.
input_typeThe type of prediction you want to make.Default value: “image_classification”;Possible values: “image_classification”, “object_detection”, “text_classification”, “structured”.

On an image classification model:

item = "path/to/file.png"

result = client.predict(my_model["id"], item)

# Or
# result = client.predict(my_model["id"], item, input_type="image_classification")

On an object detection model:

item = "path/to/file.png"

result = client.predict(my_model["id"], item, input_type="object_detection")

On a text classification model:

item = "The text you want to classify."

result = client.predict(my_model["id"], item, input_type="text_classification")

On a strctured/tabular data model:

inputs = {
    "temperature": "30",
    "day": "Tuesday"
}

item = json.dumps(inputs)

result = client.predict(my_model["id"], item, input_type="structured")

An example reply for a prediction:

{
    'id': '891ed37a-fe77-415c-a4ab-3764e68aaa40',
    'created_at': '2021-10-28T12:33:17.277236872Z',
    'update_at': '2021-10-28T12:33:17.563600762Z',d
    'name': '10789.jpg',
    'description': '',
    'prediction': 'cats',
    'confidence': 0.9846015,
    'model_id': '56086c08-c552-4159-aafa-ac4b25a64fda',
    'model_version_id': '1a38b7ab-070f-4686-9a47-4ba7a76a5168',
    'extension': 'jpg',
    'user_id': 'd7159432-f218-44ac-aebe-e5d661d62862',
    'error_reported': False,
    'error': '',
    'application_id': '80ba88dd-91e9-445f-b7ab-38ebd9ab1e63',
    'inference_host': 'image-pt-1-8-1-fa-2-3-1',
    'inference_time': '284.418389ms',
    'end_to_end_time': '',
    'dataset_item_id': '',
    'result': '',
    'inference_items': None,
    'hidden': False,
    'privacy_enabled': False
 }

Properties in more detail:

PropertyDescription
idUnique id for the model
created_atThe creation date
updated_atLast updated date
nameStores the input. For “image_classification” and “object_detection” the original filename; For ‘text_classification” and “structured” the actual text/structured input.
descriptionAdditional info. (Optional)
predictionThe model prediction or value
confidenceThe prediction confidence
model_idThe id of the model the prediction was made on.
model_version_idThe id of the model_version the prediction was made on (see ‘Model Versions’ below.
extensionThe extension of the predicted image, in case of “image_classification” and “object_detection”
user_idThe id of the user that requested the prediction
error_reportedFlag indicating whether a user has reported the prediction is/might be wrong.
errorContains the error if something went wrong.
application_idThe application_id used to make the prediction.
inference_hostThe name of the inference engine used to make the prediction.
inference_timeThe time it took to make the prediction.
end_to_end_timeInference time including upload and return (if relevant)
dataset_item_idThe id of the dataset_item that was used for the prediction. Used to evaluate datasets (see Datasets below)
ResultA string version of the object detection prediction (legacy).
inference_itemsA list of individual predictions, when using “object_detection”.
hiddenFlag indicating whether this prediction has been hidden in the Data Engine (TODO: link to Data Engine docs)
privacy_enabledFlag indicating whether this prediction was made when the model was in privacy_enabled mode.
# obsolete
# Used internally by `predict` but will be removed.
result = client.inference(my_model["id"], item, input_type="image_classification")

Model Versions #

An AI Model has one or multiple versions associated with it:

  • the current live version
  • previous versions
  • future versions

Get all model versions #

Get a list of all versions for a specific model.

versions = client.get_model_versions(my_model["id"])
ParametersDescription
model_idThe model id

Create model version #

new_version = {
    "name": "A higher accuracy achieved",
    "application_id": "878aea66-16b7-4f10-9d82-a2a92a35728a"
}

new_version = client.create_model_version(my_model["id"], new_version)
ParametersDescription
model_idThe model id
versionThe model version object

Every model version has the following properties. Note that these are partially similar to the model properties:

{
    'id': '53535e2f-4b36-4235-b8d6-ed87a5d2f323',
    'created_at': '2021-10-29T14:16:46.603463402Z',
    'update_at': '2021-10-29T14:16:46.603463402Z',
    'name': 'A higher accuracy achieved',
    'description': '',
    'model_id': '22791060-71f5-4e71-935e-28b52ef4c047',
    'classes': '',
    'accuracy': 0,
    'user_id': 'd7159432-f218-44ac-aebe-e5d661d62862',
    'can_inference': False,
    'has_logo': False,
    'logo': '',
    'config': '',
    'application_id': '878aea66-16b7-4f10-9d82-a2a92a35728a',
    'version': '',
    'version_number': 4,
    'has_ml_model': False,
    'has_onnx_model': False,
    'has_tflite_model': False,
    'has_labels_file': False,
    'dataset_version_id': '',
    'training_request_id': ''
}

Properties in more detail.

Shared with the model entity:

PropertyDescription
idUnique id for the model version
created_atThe creation date
updated_atLast updated date
nameThe model version name
descriptionThe model version description
accuracyThe model version accuracy
user_idThe user id of the model version creator
can_inferenceFlag indicating whether the model version can make predictios or not
has_logoFlag indicating whether the model has a logo or not (not used for now)
logoName and extension of the logo file (mostly for internal purpose)
configAdditional config stored in a JSON string
application_idThe application ID (see applications below)
has_ml_modelFlag indicating whether the model has a Core ML model
has_onnx_modelFlag indicating whether the model has an ONNX model
has_tflite_modelFlag indicating whether the model has a Tensorflow Lite model
has_labels_fileFlag indicating whether a file will all the labels (classes) is available

Different from the model entity.

PropertyDescription
model_idThe id of the model this version belongs to.
versionThe label of the version
version_numberAutomatically incrementing number of the version.
dataset_version_idThe id of the dataset version this model version was trained on.
training_request_idThe id of the training request used to build this model version.

Get model version #

Use the model and version id to get the full model version:

model_version = client.get_model_version(my_model["id"], new_version["id"])
ParametersDescription
model_idThe model id
version_idThe model version id

Update model version #

Update any property of the model version:

model_version["description"] = "We hit SOTA"

client.update_model_version(model_version)
ParametersDescription
model versionThe entire model version object

Delete model version #

Delete a model version:

client.delete_model_version(my_model["id"], new_version["id"])
ParametersDescription
model_idThe model id
version_idThe model version id

Upload model file for a version #

client.upload_model_version(new_version, folder="directory/to/model", filename="cats_and_dogs_v2.pkl")
ParametersDescription
versionThe entire model version object
folderName of the folder that contains the model file (without trailing ‘/'), default value “data”
filenameName of the file to be uploaded, default value “export.pkl”

Make a prediction on this version #

client.version_predict(model_version_id, item, input_type="image_classification")
ParametersDescription
model_version_idUnique id for the model version
itemThe item you wish to make a prediction on:For “image_classification” and “object_detection” specify the full file location (including directory).For “text_classification” pass in the string you would like to predict.For “structured” pass in a JSON dump of the JSON object you want to use for your prediction.
input_typeThe type of prediction you want to make.Default value: “image_classification”;Possible values: “image_classification”, “object_detection”, “text_classification”, “structured”.

On an image classification model version:

item = "path/to/file.png"

result = client.version_predict(new_version["id"], item)

# Or
# result = client.version_predict(new_version["id"], item, input_type="image_classification")

On an object detection model version:

item = "path/to/file.png"

result = client.version_predict(new_version["id"], item, input_type="object_detection")

On a text classification model version:

item = "The text you want to classify."

result = client.version_predict(new_version["id"], item, input_type="text_classification")

On a strctured/tabular data model version:

inputs = {
    "temperature": "30",
    "day": "Tuesday"
}

item = json.dumps(inputs)

result = client.version_predict(my_model["id"], item, input_type="structured")

An example reply for a prediction:

{
    'id': '891ed37a-fe77-415c-a4ab-3764e68aaa40',
    'created_at': '2021-10-28T12:33:17.277236872Z',
    'update_at': '2021-10-28T12:33:17.563600762Z',d
    'name': '10789.jpg',
    'description': '',
    'prediction': 'cats',
    'confidence': 0.9846015,
    'model_id': '56086c08-c552-4159-aafa-ac4b25a64fda',
    'model_version_id': '1a38b7ab-070f-4686-9a47-4ba7a76a5168',
    'extension': 'jpg',
    'user_id': 'd7159432-f218-44ac-aebe-e5d661d62862',
    'error_reported': False,
    'error': '',
    'application_id': '80ba88dd-91e9-445f-b7ab-38ebd9ab1e63',
    'inference_host': 'image-pt-1-8-1-fa-2-3-1',
    'inference_time': '284.418389ms',
    'end_to_end_time': '',
    'dataset_item_id': '',
    'result': '',
    'inference_items': None,
    'hidden': False,
    'privacy_enabled': False
 }

Properties in more detail:

PropertyDescription
idUnique id for the model
created_atThe creation date
updated_atLast updated date
nameStores the input. For “image_classification” and “object_detection” the original filename; For ‘text_classification" and “structured” the actual text/structured input.
descriptionAdditional info. (Optional)
predictionThe model prediction or value
confidenceThe prediction confidence
model_idThe id of the model the prediction was made on.
model_version_idThe id of the model_version the prediction was made on (see ‘Model Versions’ below.
extensionThe extension of the predicted image, in case of “image_classification” and “object_detection”
user_idThe id of the user that requested the prediction
error_reportedFlag indicating whether a user has reported the prediction is/might be wrong.
errorContains the error if something went wrong.
application_idThe application_id used to make the prediction.
inference_hostThe name of the inference engine used to make the prediction.
inference_timeThe time it took to make the prediction.
end_to_end_timeInference time including upload and return (if relevant)
dataset_item_idThe id of the dataset_item that was used for the prediction. Used to evaluate datasets (see Datasets below)
ResultA string version of the object detection prediction (legacy).
inference_itemsA list of individual predictions, when using “object_detection”.
hiddenFlag indicating whether this prediction has been hidden in the Data Engine (TODO: link to Data Engine docs)
privacy_enabledFlag indicating whether this prediction was made when the model was in privacy_enabled mode.
# obsolete
# Used internally by `predict` but will be removed.
result = client.version_inference(my_model["id"], item, input_type="image_classification")

Datasets #

Get all datasets #

Get a list of all your datasets:

datasets = client.get_datasets()

The get_datasets() method does not take any parameters.

Create dataset #

my_dataset = {
    "name": "Cats & dogs dataset",
    "description": "A dataset with labelled images of cats and dogs.",
    "multi_label": False,
    "notes": "Cats and dogs is often used as a demo dataset.",
    "default_splits": True,
    "content_type": "images"
}

my_dataset = client.create_dataset(my_dataset)
ParametersDescription
datasetThe entire dataset object

Every dataset has the following properties:

{
    'id': '2bb8b9c1-c027-44c2-b438-6d50911964bd',
    'created_at': '2021-11-02T10:52:56.386712086Z',
    'update_at': '2021-11-02T10:52:56.386712086Z',
    'deleted_at': None,
    'name': 'Cats & dogs dataset',
    'description': 'A dataset with labelled images of cats and dogs.',
    'user_id': 'd7159432-f218-44ac-aebe-e5d661d62862',
    'notes': 'Cats and dogs is often used as a demo dataset.',
    'versions': [],
    'multi_label': False,
    'default_splits': True,
    'has_logo': False,
    'logo': '',
    'content_type': 'images'
}

Properties in more detail:

PropertyDescription
idUnique id for the dataset
created_atThe creation date
updated_atLast updated date
nameThe dataset name
descriptionThe dataset description
user_idThe unique id of the dataset creator
notesMore elaborate notes about the dataset
versionsA list of all the version of the dataset (see below)
multi_labelFlag indicating whether tiems can have multiple labels
default_splitsCreate default splits (“train”, “valid”, “test”) when creating the dataset.
has_logoFlag indicating whether the dataset has a logo or not
logoName and extension of the logo file (mostly for internal purpose)
content_typeType of items in the dataset

Get dataset #

dataset_id = "2bb8b9c1-c027-44c2-b438-6d50911964bd"

my_dataset = client.get_dataset(dataset_id)
ParametersDescription
dataset_idThe dataset id

Update dataset #

my_dataset["notes"] += "~25k labelled images of cats and dogs; 22500 for training, 2000 for validation."

client.update_dataset(my_dataset)
ParametersDescription
datasetThe entire dataset object

Delete dataset #

client.delete_dataset(my_dataset)
ParametersDescription
datasetThe entire dataset object
my_dataset = client.upload_dataset_logo(my_dataset["id"], folder="directory/to/logo", filename="logo_filename.jpg")
ParametersDescription
dataset_idUnique id for the dataset
folderName of the folder that contains the logo file (without trailing ‘/'), default value “data”
filenameName of the file to be uploaded, default value “logo.jpg”. Supported formats: jpg, jpeg, png.
client.get_dataset_logo(my_dataset)
ParametersDescription
modelThe entire model object

Dataset Versions #

A dataset can have multiple versions.

Get all dataset versions #

dataset_versions = client.get_dataset_versions(my_dataset["id"])
ParametersDescription
dataset_idThe dataset id

Create dataset version #

new_dataset_version = {
    "name": "v2",
    "description": "Even more images of dogs and cats"
}

new_dataset_version = client.create_dataset_version(my_dataset["id"], new_dataset_version)
ParametersDescription
dataset_idThe dataset id
dataset_versionThe dataset version object

Every dataset_version has the following properties:

{
    'id': 'd774d5bd-40fd-4c5f-8043-f1a8ed88e0d2',
    'created_at': '2021-11-02T13:17:29.24027517Z',
    'update_at': '2021-11-02T13:17:29.24027517Z',
    'deleted_at': None,
    'name': 'v2',
    'description': 'Even more images of dogs and cats',
    'labels': None,
    'user_id': 'd7159432-f218-44ac-aebe-e5d661d62862',
    'dataset_id': '2bb8b9c1-c027-44c2-b438-6d50911964bd',
    'splits': None,
    'default_split': ''
}

Properties in more detail:

PropertyDescription
idUnique id for the dataset
created_atThe creation date
updated_atLast updated date
nameThe dataset version name
descriptionThe dataset version description
user_idThe unique id of the dataset creator
labelsA list of the labels in this version
dataset_idThe id of the dataset this version belongs to
splitsA list of splits in this dataset version
default_splitThe id of split that will be shown by default

Get dataset version #

dataset_version = client.get_dataset_version(new_dataset_version["id"])
ParametersDescription
dataset_idThe dataset id

Update dataset version #

new_dataset_version["description"] = "Even more image of cats and dogs."

client.update_dataset_version(my_dataset["id"], new_dataset_version)
ParametersDescription
dataset_idThe dataset id
dataset_versionThe dataset version object

Delete dataset version #

client.delete_dataset_version(my_dataset["id"], new_dataset_version)
ParametersDescription
dataset_idThe dataset id
dataset_versionThe dataset version object

Dataset Splits #

A dataset version can have multiple splits, usually separating training, validation and test data.

Get all splits #

splits = client.get_dataset_splits(my_dataset["id"], new_dataset_version["id"])
ParametersDescription
dataset_idThe dataset id
dataset_version_idThe dataset version id

Create split #

new_split = {
    "name": "train",
    "description": "training data for our model to learn from" 
}

new_split = client.create_dataset_split(my_dataset["id"], new_dataset_version["id"], new_split)
ParametersDescription
dataset_idThe dataset id
dataset_version_idThe dataset version id
splitThe split object

Every dataset split has the following properties:

{
    'id': '3faa6ddb-f5d6-4dda-92a9-beb383126072',
    'created_at': '2021-11-02T14:02:15.303555455Z',
    'updated_at': '2021-11-02T14:02:15.303555455Z',
    'name': 'train',
    'description': 'training data for our model to learn from',
    'user_id': 'd7159432-f218-44ac-aebe-e5d661d62862',
    'version_id': 'd774d5bd-40fd-4c5f-8043-f1a8ed88e0d2'
}
 

Properties in more detail:

PropertyDescription
idUnique id for the dataset split
created_atThe creation date
updated_atLast updated date
nameThe dataset split name
descriptionThe dataset split description
user_idThe unique id of the dataset split creator
version_idThe unique id of the dataset version the split belongs to

Get split #

my_split = client.get_dataset_split(my_dataset["id"], new_dataset_version["id"], new_split["id"])
ParametersDescription
dataset_idThe dataset id
dataset_version_idThe dataset version id
split_idThe split id

Update split #

my_split["description"] = "Training data"

client.update_dataset_split(my_dataset["id"], new_dataset_version["id"], my_split)
ParametersDescription
dataset_idThe dataset id
dataset_version_idThe dataset version id
splitThe split object

Delete split #

client.delete_dataset_split(my_dataset["id"], new_dataset_version["id"], my_split)
ParametersDescription
dataset_idThe dataset id
dataset_version_idThe dataset version id
splitThe split object

Dataset Labels #

A dataset version can have multiple labels.

Get all labels #

labels = client.get_dataset_labels(my_dataset["id"], new_dataset_version["id"])
ParametersDescription
dataset_idThe dataset id
dataset_version_idThe dataset version id

Create label #

new_label = {
    "name": "Birds",
    "description": "Adding birds to the mix"
}

new_label = client.create_dataset_label(my_dataset["id"], new_dataset_version["id"], new_label)
ParametersDescription
dataset_idThe dataset id
dataset_version_idThe dataset version id
labelThe label object

Every dataset split has the following properties:

{
    'id': 'ea3be37d-ce48-44bc-4435-39d88a47a2d6',
    'created_at': '2021-11-02T14:24:06.187758716Z',
    'updated_at': '2021-11-02T14:24:06.187758716Z',
    'name': 'Birds',
    'description': 'Adding birds to the mix',
    'user_id': 'd7159432-f218-44ac-aebe-e5d661d62862',
    'version_id': 'd774d5bd-40fd-4c5f-8043-f1a8ed88e0d2',
    'color': '',
    'index': 2
}

Properties in more detail:

PropertyDescription
idUnique id for the dataset split
created_atThe creation date
updated_atLast updated date
nameThe dataset label name
descriptionThe dataset label description
user_idThe unique id of the dataset label creator
version_idThe unique id of the dataset label the split belongs to
colorThe hex code for the color to be used/associated with this label
indexMake sure we can always sort the labels in the same sequence (handled automatically)

Get label #

my_label = client.get_dataset_label(my_dataset["id"], new_dataset_version["id"], new_label["id"])
ParametersDescription
dataset_idThe dataset id
dataset_version_idThe dataset version id
label_idThe label id

Update label #

my_label["color"] = "#00ff00"

client.update_dataset_label(my_dataset["id"], new_dataset_version["id"], my_label)
ParametersDescription
dataset_idThe dataset id
dataset_version_idThe dataset version id
labelThe label object

Delete label #

client.delete_dataset_label(my_dataset["id"], new_dataset_version["id"], my_label)
ParametersDescription
dataset_idThe dataset id
dataset_version_idThe dataset version id
labelThe label object

Dataset Items #

A dataset version can contain many dataset items. Items are used to access and store the actual items in your dataset version such as images or text.

Get dataset items #

items = client.get_dataset_items(my_dataset["id"], new_dataset_version["id"])
ParametersDescription
dataset_idThe dataset id
dataset_version_idThe dataset version id
paramsAdditional query parameters:Default value: NoneAccepted parameters: pageSize: Integer value; Number of items to be returned. Default value: 10. pageCount: Integer value: The number of the page you want to view. onlyUnlabelled: Boolean: If True only return items that are not labelled/annotated.labelId: string; Only return items that are labelled with this id. splitId: string; Only return items that are part of this split.
# All params are optional, but here we combine them together for demo purposes. 
params = {
    "onlyUnlabelled": False,
    "pageSize": 25,
    "pageCount": 0,
    "labelId": new_label["id"],
    "splitId": my_split["id"]
}

client.get_dataset_items(my_dataset["id"], new_dataset_version["id"], params)

Create dataset item #

item = {
    "name": "An optional name",
}

item = client.create_dataset_item(my_dataset["id"], new_dataset_version["id"], item)
ParametersDescription
dataset_idThe dataset id
dataset_version_idThe dataset version id
itemThe dataset item object

Every dataset item has the following properties:

{
     'id': 'e92dcc17-cbc2-4a1e-a332-4c62a3232ede',
     'created_at': '2021-11-02T16:07:15.88589873Z',
     'updated_at': '2021-11-02T16:07:15.88589873Z',
     'name': 'An optional name',
     'description': '',
     'user_id': 'd7159432-f218-44ac-aebe-e5d661d62862',
     'text': '',
     'splits': None,
     'annotations': None,
     'extension': ''
 }

Properties in more detail:

PropertyDescription
idUnique id for the dataset item
created_atThe creation date
updated_atLast updated date
nameAn optional name
descriptionThe dataset item description
user_idThe unique id of the dataset label creator
splitsThe list of dataset splits the dataset item belongs to
annotationsThe list of annotations for the dataset item
extensionThe extension for the dataset item.

Get dataset item #

item = client.get_dataset_item(my_dataset["id"], new_dataset_version["id"], item["id"])
ParametersDescription
dataset_idThe dataset id
dataset_version_idThe dataset version id
item_idThe dataset item id

Update dataset item #

item["description"] = "A better description"

client.update_dataset_item(my_dataset["id"], new_dataset_version["id"], item)
ParametersDescription
dataset_idThe dataset id
dataset_version_idThe dataset version id
itemThe dataset item object

Delete dataset item #

client.delete_dataset_item(my_dataset["id"], new_dataset_version["id"], item)
ParametersDescription
dataset_idThe dataset id
dataset_version_idThe dataset version id
itemThe dataset item object

Upload dataset item image #

client.upload_dataset_item_image(my_dataset["id"], new_dataset_version["id"], item["id"], folder="directory/to/item", filename="item_filename.jpg")
ParametersDescription
dataset_idThe dataset id
dataset_version_idThe dataset version id
itemThe dataset item object

Download dataset item image #

download_location = f"{item['id']}.{item['extension']}"

client.download_dataset_item_image(my_dataset["id"], new_dataset_version["id"], item["id"], download_location, thumbnail=False)
ParametersDescription
dataset_idThe dataset id
dataset_version_idThe dataset version id
item_idThe dataset item id
download_locationThe full location and filename of where to save the item image.
thumbnailFlag indicating whether to download the full image or its thumbnail.

Export Dataset Version #

client.download_dataset(my_dataset["id"], new_dataset_version["id"], split_id="", extract_to_dir="data", download_file="dataset.zip", remove_download_file=True)
ParametersDescription
dataset_idThe dataset id
dataset_version_idThe dataset version id
split_id(Optional) Specify the split_id if you only want to download that dataset split.
extract_to_dirThe directory to extract to. Default value: “data”
download_fileThe name of the download file. Default value: “dataset.zip”
remove_download_fileFlag indicating whether to remove or keep the downloaded zip file. Default value: True (= remove)

A note on the exported formats.

Image classification #

The .zip file contains a folder for every dataset split: “train”, “valid”, “test”.

Every dataset split folder contains a ‘label-named’ folder for every label: “cats”, “dogs”.

Every label folder contains all the images of the dataset items for the given label in the given dataset split.

Dataset items are named by the “name” and “extension” property of the dataset item. If the “name” property is empty, the “id” is used to name the file.

Dataset items that have no label, will be added to the split folder to which they belong.

.
+-- train/
|  +-- cats/
|     +-- cat1.jpg
|     +-- cat12.jpeg
|  +-- dogs/
|     +-- dog2.jpg
|     +-- dog4.png
|  +-- cat17.jpeg
|  +-- dog15.jpg
+-- valid/
|  +-- cats/
|     +-- cat4.jpg
|     +-- cat8.jpg
|  +-- dogs/
|     +-- dog9.jpg
|     +-- dog14.png
+-- test/
|  +-- cats/
|     +-- cat90.jpg
|     +-- cat34.jpeg
|  +-- dogs/
|     +-- dog81.jpg
|     +-- dog98.png

Text classification #

The .zip file contains a folder for every dataset split: “train”, “test”, “unsup”.

Every dataset split folder contains a ‘label-named’ folder for every label: “pos”, “neg”.

Every label folder contains all the images of the dataset items for the given label in the given dataset split.

Dataset items are named by the “name” and “extension” property of the dataset item. If the “name” property is empty, the “id” is used to name the file.

Dataset items that have no label, will be added to the split folder to which they belong.

.
+-- train/
|  +-- pos/
|     +-- 1.txt
|     +-- 3.txt
|  +-- neg/
|     +-- 2.txt
|     +-- 4.txt
+-- test/
|  +-- pos/
|     +-- 5.txt
|     +-- 7.txt
|  +-- neg/
|     +-- 6.txt
|     +-- 8.txt
+-- unsup/
|     +-- 13.txt
|     +-- 14.txt

Object detection #

Object detection datasets are exported in YOLO format.

For every dataset split a dataset_split_name.txt file gets created containing all the filenames for that dataset split.

Every dataset item will have an image and a txt file associated with it. The txt file contains a list of annotations in Yolo format: label_index relative_x relative_y relative_width relative_height.

The .names file contains the list of labels, where the index corresponds to the label_index in the annotation .txt files.

The config.json file contains a contains a json object with the color for every label.

.
+-- train.txt
+-- valid.txt
+-- test.txt
+-- 1.jpg
+-- 1.txt
+-- 3.jpg
+-- 3.txt
+-- ...
+-- dataset_version_id.names
+-- config.json

Import Dataset Version #

For image_classification and text_classification, you can upload a .zip of the dataset directly.

The zip file should be structured in the following way, for image_classification:

.
+-- train/
|  +-- cats/
|     +-- cat1.jpg
|     +-- cat12.jpeg
|  +-- dogs/
|     +-- dog2.jpg
|     +-- dog4.png
|  +-- cat17.jpeg
|  +-- dog15.jpg
+-- valid/
|  +-- cats/
|     +-- cat4.jpg
|     +-- cat8.jpg
|  +-- dogs/
|     +-- dog9.jpg
|     +-- dog14.png
+-- test/
|  +-- cats/
|     +-- cat90.jpg
|     +-- cat34.jpeg
|  +-- dogs/
|     +-- dog81.jpg
|     +-- dog98.png

and for text_classification:

.
+-- train/
|  +-- pos/
|     +-- 1.txt
|     +-- 3.txt
|  +-- neg/
|     +-- 2.txt
|     +-- 4.txt
+-- test/
|  +-- pos/
|     +-- 5.txt
|     +-- 7.txt
|  +-- neg/
|     +-- 6.txt
|     +-- 8.txt
+-- unsup/
|     +-- 13.txt
|     +-- 14.txt

This way all splits, labels, items, and annotations will automatically be imported.

client.upload_dataset_version(my_dataset["id"], new_dataset_version["id"], folder="data", filename="dataset.zip")
ParametersDescription
dataset_idThe dataset id
dataset_version_idThe dataset version id
folderThe folder that contains the .zip file. Default value: “data”
filenameThe name of the upload file. Default value: “dataset.zip”

Annotations #

Manage annotations by adding, updating, and deleting them in the annotations list of the dataset item

Get Annotations #

item["annotations"]

Create annotation #

annotation = {
    "label_id": "1d57e9a8-8993-4de6-a69d-57192b27ab0e",
    "coordinates": "0.318984375 0.24546875 0.07359375 0.0665625"
}

item["annotations"].append(annotation)

item = client.update_dataset_item(my_dataset["id"], new_dataset_version["id"], item)

The dataset item has a list of annotations. Every annotation has the following properties:

{
    'id': '1d57e9a8-8993-4de6-a69d-57192b27ab0e',
    'created_at': '2021-11-03T13:15:17.237588533Z',
    'updated_at': '2021-11-03T13:15:17.237588533Z',
    'label_id': 'ea3be37d-ce48-44bc-4435-39d88a47a2d6',
    'item_id': 'e92dcc17-cbc2-4a1e-a332-4c62a3232ede',
    'coordinates': '0.318984375 0.24546875 0.07359375 0.0665625'
}

Properties in more detail.

PropertyDescription
idUnique id for the annotation
created_atThe creation date
updated_atLast updated date
label_idThe label Id for the annotation
item_idThe item Id for the annotation
coordinates(Object detection) The Yolo coordinates

Get annotation #

item["annotations"] = [{'id': '5308a7c7-b137-46e0-b718-08d1feb78b6d',
   'created_at': '2021-11-03T13:28:52.187604387Z',
   'updated_at': '2021-11-03T13:28:52.187604387Z',
   'label_id': 'ea3be37d-ce48-44bc-4435-39d88a47a2d6',
   'item_id': 'e92dcc17-cbc2-4a1e-a332-4c62a3232ede',
   'coordinates': ''},
  {'id': '5a44653c-89af-4317-abbe-fc0ae8d1b8bf',
   'created_at': '2021-11-03T13:28:52.187916713Z',
   'updated_at': '2021-11-03T13:28:52.187916713Z',
   'label_id': '1d57e9a8-8993-4de6-a69d-57192b27ab0e',
   'item_id': 'e92dcc17-cbc2-4a1e-a332-4c62a3232ede',
   'coordinates': '0.318984375 0.24546875 0.07359375 0.0665625'}]
item["annotations"][0]

Update annotation #

item["annotations"][0]["coordinates"] = "0.4234 0.12425 0.075634 0.0548990"

item = client.update_dataset_item(my_dataset["id"], new_dataset_version["id"], item)

Delete annotation #

del(item["annotations"][0])
client.update_dataset_item(my_dataset["id"], new_dataset_version["id"], item)

Applications #

SeeMe.ai supports multiple types of AI models, frameworks, and framework versions. To access, manage, and describe these, we use applications:

Get supported applications #

Print a list of the applications in your SeeMe.ai client:

client.applications

Every application has the following properties:

{
    'id': '878aea66-16b7-4f10-9d82-a2a92a35728a',
    'created_at': '2021-10-25T17:57:14.323965Z',
    'updated_at': '2021-10-25T17:57:14.323965Z',
    'base_framework': 'pytorch',
    'base_framework_version': '1.10.0',
    'framework': 'fastai',
    'framework_version': '2.5.3',
    'application': 'image_classification',
    'inference_host': 'image-pt-1-10-0-fa-2-5-3',
    'can_convert_to_onnx': True,
    'can_convert_to_coreml': True,
    'can_convert_to_tensorflow': True,
    'can_convert_to_tflite': True,
    'has_labels_file': True,
    'inference_extensions': 'pkl'
}

Properties in more detail.

PropertyDescription
idUnique id for the application
created_atThe creation date
updated_atLast updated date
frameworkThe framework used to train the model
framework_versionThe framework version used to train the model
base_frameworkThe base framework used by the framework
base_framework_versionThe base framework version used by the framework
applicationThe type of application: “image_classification”, “object_detection”, “text_classification”, “structured”.
inference_hostThe internal host of the inference engine (if not used at the edge)
can_convert_to_onnxFlag indicating whether uploaded models can automatically be converted to ONNX
can_convert_to_coremlFlag indicating whether uploaded models can automatically be converted to Core ML
can_convert_to_tensorflowFlag indicating whether uploaded models can automatically be converted to Tensorflow (mostly for further conversion to Tensorflow Lite for example)
can_convert_to_tfliteFlag indicating whether uploaded models can automatically be converted to Tensorflow Lite
has_labels_fileFlag indicating whether there is file with all the labels available
inference_extensionsThe list of files with these extensions that need to be uploaded before the model can perform predictions/inferences.

You can update the local list of applications by:

client.applications = client.get_applications()

Get application id #

Before you can upload and use your model to make predictions, you need to add an application_id:

from torch import __version__ as torch_version
from fastai import __version__ as fastai_version

# Get the application_id for your framework (version).
application_id = client.get_application_id(
    base_framework="pytorch",
    framework="fastai",
    base_framework_version=torch_version,
    framework_version=fastai_version,
    application="image_classification"
)

If your combination is not supported, you will get a NotImplementedError.

Feedback #

For questions, feedback, corrections: contact jan.vandepoel@seeme.ai.