Spike's use cases for Orchest include:
Spike is a leading machine learning and data innovation company based in Chile, servicing a broad array of clients throughout South America. With a deep bench of machine learning experts they're changing how companies uncover insight from their data. Spike is delivering scalable solutions to problems requiring new insight in demanding environments with modern data science tooling and custom approaches. As of September 2021, Spike counts the largest and most innovative companies across Healthcare, Airlines, Mining, Telco, and Energy as their clients (P.S. If you want to review their innovative work, check out their Medium).
Spike's challenge was to empower eventual future data science hires at their client with ready to use models & infrastructure whilst also delivering models at production as a managed service until their client properly staffed up. When evaluating the landscape, Spike considered alternatives such as AWS Lambda, KNIME, Kubeflow and Airflow in combination with Papermill but ultimately went with Orchest because it struck the perfect balance between capabilities and ease of use. The team liked Orchest’s excellent GUI, the fact that it’s open source and easy to deploy on their customer's infrastructure. The install process was easy to set up in their clients VM and scaling vertically has been great given a small team within Spike is able to stand up the clients pipelines at production. In the course of delivering the project Spike also found some additional benefit from deploying in Orchest:
'OMG! This is what Data Scientists need.'
"Before Orchest the challenge for us has always been 'How do we orchestrate various deliverables (model training/retraining, sending email with output updates, putting corresponding data in our customers database — or transforming it and putting it back)' and set up the entire data stack for clients, mostly because it is a lot of work and we are a lean team. Our Data Scientists want to use Notebooks, not tools meant for Data Engineers or SW Developers," shares Matias Aravena, Spike's lead Machine Learning Engineer. "Our Data Scientist's expertise is more related to experimental iteration of model development, not really production. MLOps is not necessarily in their toolkit. When we first tried Orchest we said 'OMG! This is what Data Scientists need.' Having Jupyterlab there is amazing, for Data Scientists it is familiar and so doesn't change their workflow at all, they can be really productive in Orchest.
It's also great that they can see a visualization of the pipeline editor because it's helpful for collaboration both internally within our teams but also with our client's teams. Our client's pipeline involves sensor data collected, moving that into Google's BigQuery, making the data available to a VM, and then from notebooks getting data included in reports delivered via email. Orchest was very easy to deploy on our customer's VM. We offer a managed service to our client for this data science project. We think Orchest brings us a lot of value not just now but in the future. The client plans on staffing up their own Data Scientists, those hires will be able to start building models from day 1 because the entire project is ready to go in Orchest."
A recent upgrade to Orchest's latest version has been helpful in empowering Spike and their client to benefit from Tensorflow and Streamlit directly within Orchest. Spike is also finding that teams within larger org's may benefit from Orchest given lack of access to required data engineering resources to get infrastructure stood up for their data science projects, "Sometimes a company's policy is heavily bureaucratic and the teams we work with can't get their job done. Maybe the infra they use is too slow, maybe we can't get access because the data we need is behind a VPN, maybe the Data Scientists at the client are regulated to have to code in Java; getting to production is hard because of all the rules. Maybe in these scenarios Orchest could help teams who are just after insight after all to move more quickly."
"I am really in love with Orchest! Getting to production is hard, but it doesn't need to be, especially for projects that are more insights driven than delivering an application. It is everything you need in one tool. For teams that are constrained for various reasons moving forward we plan on using Orchest." -Matias Aravena, Machine Learning Engineer, Spike