Skip to content

Distributed Architecture

Decision

Context

The platform parameter is inconsistent with the target device being used in the pipeline. To be consistent with OVMS we want to use the target_device parameter in the script to match the target_device setting in the OVMS config file.

Proposed Design

Update the platform parameter to match the target_device standard used by OpenVINO Model Server. This will provide clarity to the device being used for the inferencing portion of the pipeline. The following are the acceptance criteria for the change.

  • Replace platform parameter with target_device using CPU as the default device.
  • Update the docker_run script to make it run with minimal changes to the profiles.
  • Confirm that the benchmark script works with the target_device parameter update.
  • Update unit tests
  • Update documentation
  • Convert $DEVICE to $TARGET_DEVICE for internal environment variables.
  • Add option to use existing config file and not override all target_devices to support models with different target_device values.

Target Device list

Device Parameter Description Links
CPU CPU Use CPU only OVMS Parameters
GPU GPU Use default GPU OVMS Parameters
Specified GPU GPU.x Use a specific GPU. ex. GPU.0 = integrated GPU, GPU.1 = discrete Arc GPU OVMS Parameters
Mixed Contifuration MULTI:x,y Use a combination of devices for inferencing ex. MULTI:CPU,GPU.1 will use the CPU and discrete Arc GPU for inferencing OVMS Parameters
Automatic Device Selection AUTO Allow OpenVINO to automatically select the optimal device for inferencing Possibly depricated?
Automatic Device Selection AUTO Allow OpenVINO to automatically select the optimal device for inferencing Possibly depricated?
Heterogeneous Execution HETERO Allows OpenVINO to execute inference on multiple devices Heterogeneous Execution
Heterogeneous Execution Priority HETERO:x,y Allows OpenVINO to execute inference on multiple devices and set the priority of device. ex. HETERO:CPU,GPU.1 will prioritize CPU and discrete Arc GPU for inferencing Heterogeneous Execution

Applicable Repos

automated-self-checkout

Consequences

Removing the platform parameter will break any existing test and benchmark scripts. The change will clarify which device you are targeting for the inference.

References

https://docs.openvino.ai/2023.0/ovms_what_is_openvino_model_server.html https://github.com/openvinotoolkit/model_server