Writing integration tests for example pyscripts

Before a test is written it makes sense to understand how a example is structured. For this see writing-examples.md as a introduction. How tests can be structured will be demonstrated on the already created test_ gRPC_fb_addLabels test found here.

As the testing framework for python pytest is used.

The code

from typing import List

import flatbuffers
from grpc import Channel
from gRPC.images import gRPC_fb_addLabel as add_label
from gRPC.images import gRPC_pb_sendLabeledImage as add_img
from seerep.fb import image_service_grpc_fb as imageService
from seerep.util.fb_helper import createQuery
from seerep.util.fb_to_dict import SchemaFileNames, fb_flatc_dict

For the imported modules, note that the examples themselves are imported to be used in the tests. Furthermore helper functions from seerep.util.fb_helper, as well as fb_flatc_dict are imported.

def get_imgs(target_proj_uuid: str, grpc_channel: Channel) -> List:
    builder = flatbuffers.Builder(1024)
    stub = imageService.ImageServiceStub(grpc_channel)
    query = createQuery(
        builder,
        projectUuids=[target_proj_uuid],
        withoutData=True,
    )
    builder.Finish(query)
    buf = builder.Output()
    response_ls: List = list(stub.GetImage(bytes(buf)))

    return [fb_flatc_dict(buf, SchemaFileNames.IMAGE) for buf in response_ls]

This is a helper function to retrieve all images the targeted project on the targeted server has. At the end with the service call GetImage() the images are returned in a list of bytearray objects, which on the return line are converted to a python dictionary with the help of the SchemaFileNames enum. SchemaFileNames contains references to the file names of the datatypes of all the SEEREP flatbuffers types, this is required for the flatc compiler to know how to decode the object.

def test_addLabel(grpc_channel, project_setup):
    _, proj_uuid = project_setup

    # send labeled images to the server for preparation
    add_img.send_labeled_images(
        target_proj_uuid=proj_uuid, grpc_channel=grpc_channel
    )

    sent_label = add_label.add_label_raw(
        target_proj_uuid=proj_uuid, grpc_channel=grpc_channel
    )

    assert sent_label is not None

    # use a regular query to query the images from the server to check if the
    # label is the same
    all_imgs = get_imgs(proj_uuid, grpc_channel)

    for label_img_uuid, label_img in sent_label:
        img_label = [
            img
            for img in all_imgs
            if img["header"]["uuid_msgs"] == label_img_uuid
        ][0]

        # iterate through all categories of the image
        filtered_label = [
            label
            for label in img_label["labels"]
            if label["category"] == "laterAddedLabel"
        ]

        sent_labels = fb_flatc_dict(
            label_img, SchemaFileNames.DATASET_UUID_LABEL
        )["labels"]

        assert len(filtered_label) == len(sent_labels)

        for label_cat in filtered_label:
            assert label_cat in sent_labels

Next up the test function is defined and uses the fixture grpc_channel, which spins the test server up, creates a channel to that server and makes sure that the server is terminated after testing is done. The fixture project_setup creates a test project on the server and deletes that after the testing function using SEEREP server calls. Implementations of both fixtures can be found here.

After that the project_uuid of the created project is retrieved and used for the different calls to the example functions. To attach new Labels to images, it has to be ensured that images are present. This is done by utilizing the gRPC_pb_sendLabeledImage example. Then the Labels example can be used to add Labels to those images.

Following on the Labels returned by add_label_raw() are tested and compared against those now persisting on the server. get_imgs retrieves all the images from the server, each mapping of image uuid to Labels is iterated and in the list of all images the image, which matches the image uuid of the mapping, is inspected further. Only the Labels with the category laterAddedBB are relevant and therefore filtered. Lastly the sent Labels are converted to python dictionaries and compared for matching with the filtered image attached Labels.

Tips to ease development

• fb_flatc_dict() in conjunction with pytest -s command is a good option for debugging tests, as the dictionary and therefore the objects data contents can be printed that way.
• If a recursive operation has to be applied to a dictionary boltons remap() function can be used, like in test_gRPC_fb_createGeodeticCoordProject.py.
• If a test should contain a lot of variations in the components of a datatype a look here could simplify things