Listed here are the changes between each release of SmartSim and SmartRedis.
Jump to SmartRedis Changelog
Released on June 24, 2022
Description: This release of SmartSim introduces a new experimental feature to help make SmartSim workflows more portable: the ability to run simulations models in a container via Singularity. This feature has been tested on a small number of platforms and we encourage users to provide feedback on its use.
We have also made improvements in a variety of areas: new utilities to load scripts and machine learning models into the database directly from SmartSim driver scripts and install-time choice to use either KeyDB or Redis for the Orchestrator. The RunSettings API is now more consistent across subclasses. Another key focus of this release was to aid new SmartSim users by including more extensive tutorials and improving the documentation. The docker image containing the SmartSim tutorials now also includes a tutorial on online training.
Documentation and tutorials
General improvements and bug fixes
Set models and scripts at the driver level (SmartSim-PR185)
Optionally use KeyDB for the orchestrator (SmartSim-PR180)
Fix the handling of LSF gpus_per_shard (SmartSim-PR164)
Fix error when re-running smart build` (SmartSim-PR165)
Fix generator hanging when tagged configuration variables are missing (SmartSim-PR177)
Released on Feb 11, 2022
Description: In this release SmartSim continues to promote ease of use. To this end SmartSim has introduced new portability features that allow users to abstract away their targeted hardware, while providing even more compatibility with existing libraries.
A new feature, Co-located orchestrator deployments has been added which provides scalable online inference capabilities that overcome previous performance limitations in seperated orchestrator/application deployments. For more information on advantages of co-located deployments, see the Orchestrator section of the SmartSim documentation.
The SmartSim build was significantly improved to increase
customization of build toolchain and the
line inferface was expanded.
Additional tweaks and upgrades have also been made to ensure an optimal experience. Here is a comprehensive list of changes made in SmartSim 0.4.0.
Emphasize Driver Script Portability:
Expand Machine Learning Library Support:
ML backend versions updated with expanded support for multiple versions (SmartSim-PR122)
Launch Ray internally using
Add Ray cluster setup and deployment to SmartSim (SmartSim-PR50)
Expand Launcher Setting Options:
Deprecations and Breaking Changes
Orchestrator classes combined into single implementation for portability (SmartSim-PR139)
TOML configuration option removed in favor of environment variable approach (SmartSim-PR122)
General Improvements and Bug Fixes:
Abstract away non-user facing implementation details (SmartSim-PR122)
Add various dimensions to the CI build matrix for SmartSim testing (SmartSim-PR130)
Add missing functions to LSFSettings API (SmartSim-PR113)
Add RedisAI checker for installed backends (SmartSim-PR137)
Fix LSFLauncher and LSFOrchestrator (SmartSim-PR86)
Fix over greedy Workload Manager Parsers (SmartSim-PR95)
Fix Slurm handling of comma-separated env vars (SmartSim-PR104)
Fix internal method calls (SmartSim-PR138)
Released on August 10, 2021
Upgraded RedisAI backend to 1.2.3 (SmartSim-PR69)
PyTorch 1.7.1, TF 2.4.2, and ONNX 1.6-7 (SmartSim-PR69)
LSF launcher for IBM machines (SmartSim-PR62)
Improved code coverage by adding more unit tests (SmartSim-PR53)
Orchestrator methods to get address and check status (SmartSim-PR60)
Added Manifest object that tracks deployables in Experiments (SmartSim-PR61)
Improved IP address aquisition (SmartSim-PR72)
Binding database to network interfaces
Released on May 5, 2021
This release was dedicated to making the install process
easier. SmartSim can be installed from PyPi now and the
smart cli tool makes installing the machine learning
runtimes much easier.
Released on April 1, 2021
initial 0.3.0 (first public) release of SmartSim
Released on June 24, 2022
Version 0.3.1 adds new functionality in the form of DataSet aggregation lists for pipelined retrieval of data, convenient support for multiple GPUs, and the ability to delete scripts and models from the backend database. It also introduces multithreaded execution for certain tasks that span multiple shards of a clustered database, and it incorporates a variety of internal improvements that will enhance the library going forward.
Implemented multithreaded execution for parallel dataset list retrieval on clustered databases. The number of threads devoted for this purpose is controlled by the new environment variable SR_THERAD_COUNT. The value defaults to 4, but may be any positive integer or special value zero, which will cause the SmartRedis runtime to allocate one thread for each available hardware context. (PR251) (PR246)
Added API calls for all clients to delete models and scripts from the backend database. (PR240) New commands are:
Updated the use of backend RedisAI API calls to discontinue use of deprecated methods for model selection (AI.MODELSET) and execution (AI.MODELRUN) in favor of current methods AI.MODELSTORE and AI.MODELEXECUTE, respectively. (PR234)
SmartRedis will no longer call the C runtime method srand() to ensure that it does not interfere with random number generation in client code. It now uses a separate instance of the C++ random number generator. (PR233)
Updated the way that the Fortran enum_kind type defined in the fortran_c_interop module is defined in order to better comply with Fortran standard and not interfere with GCC 6.3.0. (PR231)
Corrected the spelling of the word “command” in a few error message strings. (PR221)
SmartRedis now requires a CMake version 3.13 or later in order to utilize the add_link_options CMake command. (PR217)
Updated and improved the documentation of the SmartRedis library. In particular, a new SmartRedis Integration Guide provides an introduction to using the SmartRedis library and integrating it with existing software. (PR261) (PR260) (PR259) (SSPR214)
Added clustered Redis testing to automated GitHub check-in testing. (PR239)
Updated the SmartRedis internal API for building commands for the backend database. (PR223) This change should not be visible to clients.
The SmartRedis example code is now validated through the automated GitHub checkin process. This will help ensure that the examples do not fall out of date. (PR220)
Added missing copyright statements to CMakeLists.txt and the SmartRedis examples. (PR219)
Updated the C++ test coverage to ensure that all test files are properly executed when running “make test”. (PR218)
Fixed an internal naming conflict between a local variable and a class member variable in the DataSet class. (PR215) This should not be visible to clients.
Updated the internal documentation of methods in SmartRedis C++ classes with the override keyword to improve compliance with the latest C++ standards. (PR214) This change should not be visible to clients.
Renamed variables internally to more cleanly differentiate between names that are given to clients for tensors, models, scripts, datasets, etc., and the keys that are used when storing them in the backend database. (PR213) This change should not be visible to clients.
Released on Febuary 11, 2022
Includes changes to C and Fortran function prototypes that are not backwards compatible
Includes changes to error class names and enum type names that are not backwards compatible
poll_datasetfunctionality to all SmartRedis clients (PR184)
Due to other breaking changes made in this release, applications using methods other than
poll_datasetto check for the existence of a dataset should now use
Add environment variables to control client connection and command timeout behavior (PR194)
Add AI.INFO command to retrieve statistics on scripts and models via Python and C++ clients (PR197)
Create a Dockerfile for SmartRedis (PR180)
redis-plus-plusversion to 1.3.2 (PR162)
Extend inverse CRC16 prefixing to all hash slots (PR161)
Improve backend dataset representation to enable performance optimization (PR195)
Simplify SmartRedis build proccess (PR189)
Fix zero-length array transfer in Fortran
Released on August, 5, 2021
Improved tensor memory management in the Python client (PR70)
Improved metadata serialization and removed protobuf dependency (PR61)
Added unit testing infrastructure for the C++ client (PR96)
Added copy, rename, and delete tensor and DataSet commands in the Python client (PR66)
Upgrade to RedisAI 1.2.3 (PR101)
Add Redis INFO command execution to the Python client (PR83)
Add Redis CLUSTER INFO command execution to the Python client (PR105)
Released on May 5, 2021
Released on April 1, 2021
Initial 0.1.0 release of SmartRedis