ElixirConf 2023 - Sean Moriarity - MLOps in Elixir: Simplifying traditional MLOps with Elixir
MLOps (machine learning operations) and, more recently LLMOps (large language model operations) are trendy terms for the paradigms and practices of deploying machine learning models. The MLOps ecosystem is a forest of tools and so-called best practices. Entire companies are built to support the machine learning lifecycle. However, with Elixir, we can greatly simplify this process without much effort. This talk will show you how to do MLOps in Elixir.
- With Nx Servings, you can easily batch multiple inference requests.
- This is very helpful because of a lot of reasons. One of the main one being that there’s a lot of overhead in transferring the data from CPU to GPU. CPU & GPU have separate RAM. A variable you store in CPU memory cannot be directly accessed in GPU kernel. We need to copy it there. With batching, you dont have to do the copying again & again!