COG

Cog is an open-source, command-line tool designed to package machine learning (ML) models. It offers a consistent environment to develop, train and collaborate on ML models. When the model is trained, it can be shared or deployed by baking it into a Docker image that runs a standard HTTP API.

Cog also has some special features compared to ordinary Docker:

• Automatic Docker image. All you need is a configuration file to define the environment, and Cog will generate a Dockerfile with the right configurations and best practices.
• Automatic HTTP service. Cog will generate a HTTP service from the model definition, eliminating the need to write a Flask server.
• No more CUDA hell. Cog knows which CUDA/cuDNN/PyTorch/Tensorflow/Python combinations are compatible and will make the right selections for you.

Source: https://github.com/replicate/cog