What is a data pool?

A data pool is a fundamental building block of Lentiq. It is to a degree a "mini data lake" designed to be used by a small team of people to share files, tables, notebooks, models etc. The team can also choose it's own stack of applications to use such as Spark or Ray.

It uses docker containers to run notebooks, additional applications, reusable code blocks that run as part of a workflow etc. The compute layer (the Kubernetes cluster behind the data pool) is separated from the storage layer so both can scale independently as needed.