The key to reliability - Testing in the field of ML-Ops [PyCon DE & PyData Berlin 2024]

🔊 Recorded at PyCon DE & PyData Berlin 2024, 23.04.2024
2024.pycon.de/...
🎓 Watch the experts from idealo.de share their innovative approach and lessons learned from 4 years of experience in testing machine learning components in a dynamic environment.
Speakers:
Gunar Maiwald, Tobias Senst
Description:
Gunar Maiwald, a Senior Machine Learning Engineer at idealo.de, and Tobias Senst, also a Senior Machine Learning Engineer, delivered a talk on the importance of testing in ML-Ops. They emphasized the growing significance of testing in the development and operation of machine learning-based components. Gunar and Tobias shared insights from their experience in operating idealo's cataloging system, a platform that handles millions of product offerings from thousands of shops.
The speakers highlighted that machine learning components should be viewed as part of a complex domain, particularly in an event-driven asynchronous architecture. They stressed the necessity for stable software development and operations in tandem with traditional software components. The talk covered various aspects of a holistic testing approach, including machine learning lifecycle, traditional software testing like unit tests and code coverage, and specific challenges in the ML domain such as testing training pipelines and deployment of inference endpoints.
Gunar and Tobias discussed the critical role of logging and monitoring for ensuring safe operations. They showcased their testing strategy developed over 4 years at idealo, featuring tools like PyTest, CDK, Pactman, AWS Sagemaker, Github Actions, OpenSearch Kibana, and Grafana. Attendees gained valuable insights into effective testing methodologies for machine learning systems from the speakers' practical experience and expertise in the field.
⭐️ About PyCon DE & PyData Berlin:
The PyCon DE & PyData conference unite the Python, AI, and data science communities, offering a unique platform for collaboration and innovation. The PyCon DE & PyData Berlin 2024 conference, hosted in partnership with the local Berlin PyData chapter, provided an exceptional experience, fostering deeper connections within the Python community while showcasing advancements in AI and data science. Attendees enjoyed a diverse and engaging program, solidifying the event as a highlight for Python and AI enthusiasts nationwide.
Follow us:
• LinkedIn: / 28908640
• X: www.x.com/pyconde
• X: www.x.com/pyda...
Links:
• Conference website: pycon.de
• Related sessions: 2024.pycon.de/p...
The conference is organized by
• Python Softwareverband e.V.: pysv.org
• NumFOCUS Inc.: numfocus.org
• Pioneers Hub gemeinnützige GmbH: pioneershub.org
If you enjoyed this session, please like, comment, and subscribe to our channel for more insightful talks and discussions.
Share this video with your network to spread the knowledge!
Hashtags:
#Python #PyConDE #PyData #OpenSource #AI #DataScience #MachineLearning #SoftwareDevelopment #LLMs #Community
Acknowledgements:
Special thanks to all the volunteers and sponsors who made this event possible.
About:
Python Softwareverband e.V.:
PySV is a non-profit that promotes the use and development of Python in Germany through events, education, and advocacy, fostering an open Python community.
NumFOCUS Inc.
supports open-source scientific computing by providing financial and logistical support to key projects like NumPy and Jupyter, promoting sustainable development and collaboration.
Pioneers Hub gemeinnützige GmbH:
is a non-profit fostering innovation in AI and tech by connecting experts and promoting knowledge exchange through events and collaborative initiatives.