Adopting “as a service” for Data Science

I recently wrote a blog about Jupyter Notebook as a service and this brought up lots of questions from my peers. I would like to take a moment to address one that I think is important. What business challenges initiated this project?

What did we solve for customer zero? We solved the challenge where central data sets were shared across hundreds of resources between multiple Geos with little control, and secondly low utilization on central infrastructure and overutilization on user infrastructure. They had challenges where many researchers were straining the system as increasing demand to access data, and conduct work using datasets. There was a need to develop new tools on a platform that was standardised and scalable for the future. Some researchers were doing all the coding using local deployments of Jupiter on laptop resources to run research, and because it was running slow, they were asking for more expensive laptops. They were copying the research data and source code down to their laptop so they could execute against it. Which meant you had data and source code everywhere, and then out the door. Which meant security was also a concern.

When laptops didn't cut it, People were going out and buying server nodes sticking them under desks and using that as their own HPC research environment or software dev environment, or analytics environment because it was nice and close to them. They could control it and they got their cloud-like experience without having to go back to Central IT.