Orchestrator

The Orchestrator is an in-memory database that is launched prior to all other entities within an Experiment. The Orchestrator can be used to store and retrieve data during the course of an experiment. In order to stream data into or receive data from the Orchestrator, one of the SmartSim clients (SmartRedis) has to be used within a Model.

Combined with the SmartRedis clients, the Orchestrator is capable of hosting and executing AI models written in Python on CPU or GPU. The Orchestrator supports models written with TensorFlow, Pytorch, TensorFlow-Lite, or models saved in an ONNX format (e.g. sci-kit learn).

The Orchestrator API is implemented for each launcher that SmartSim supports.

• SlurmOrchestrator for Slurm managed systems

• CobaltOrchestrator for Cobalt managed systems

• PBSOrchestrator for PBSPro managed systems.

• LSFOrchestrator for LSF managed systems.

The base Orchestrator class can be used for launching Redis locally on single node workstations or laptops.

Redis

The Orchestrator is built on Redis. Largely, the job of the Orchestartor is to create a Python reference to a Redis deployment so that users can launch, monitor and stop a Redis deployment on workstations and HPC systems.

Redis was chosen for the Orchestrator because it resides in-memory, can be distributed on-node as well as across nodes, and provides low-latency data access to many clients in parallel. The Redis ecosystem was a primary driver as the Redis module system provides APIs for languages, libraries, and techniques used in Data Science. In particular, the Orchestrator relies on RedisAI to provide access to Machine Learning runtimes.

At its core, Redis is a key-value store. This means that put/get semantics are used to send messages to and from the database. SmartRedis clients use a specific hashing algorithm, CRC16, to ensure that data is evenly distributed amongst all database nodes. Notably, a user is not required to know where (which database node) data or Datasets (see Dataset API) are stored as the SmartRedis clients will infer their location for the user.