Data Overload Vol. 11

Welcome to Data Overload Vol. 11, the monthly newsletter with company updates from Orchest!

Working on the new navigation 🎨

We already started working on implementing the new navigation concept that our designer Nick shared with the Orchest insiders in our #beta-testers channel. Expect a much cleaner and intuitive user interface very soon! If you want to get a sneak peek, make sure to join and participate in our Slack!

Bringing Orchest to all of Kubernetes 🐳

Our objective is that you are able to install Orchest in all major Kubernetes implementations and Docker runtimes, and our Backend Cloud Engineer Navid has started working on this front. We are also crafting some documentation changes so that selecting the installation procedure that best suits your needs is a no-brainer. Stay tuned!

Spreading the love at PyData London 🇬🇧

After a short trip to Vilnius for PyCon Lithuania, our Data Scientist Advocate Juan Luis went to PyData London to give a talk titled "Beyond pandas: The great Python dataframe showdown" that sparked lots of interesting discussions. All the materials are available so you can import them into your Orchest instance, and while you wait for the recording you can read some more background in our blog post lightning-fast queries with Polars.

Thank you!

Our users keep sending us feedback about the product and the early ideas we share with them. Thank you to everyone that helps us make Orchest a great tool!

- Juan Luis Cano (Data Scientist Advocate)

Product updates

🔔 Notifications for job failures

Now you can set up an outgoing webhook that notifies you if a job fails. Try it out!

🗄️ Load job pipeline parameters from JSON files

Jobs can now be parametrized using JSON files in your project tree.

What we're reading

• I wrote one of the fastest DataFrame libraries (Polars) (blog post)
  
  We admit we have fallen in love with Polars a bit, and this blog post is a good summary of the reasons for it: Ritchie Vink, the project creator, goes over some of the optimizations and tricks he used to make Polars one of the fastest dataframe libraries out there (hand-in-hand with DuckDB, which we are also following closely). Totally recommended!