Design Ideas for Frustrated ML Engineers

Simple ideas to solve complex problems.

The Night Café, 1888 by Vincent Van Gogh (Source)
ML System Diagram
A generic diagram of the production ML System. (Author)

Keep it simple

Simple python project repo for ML
Start with the simplest solution. (Author)
  1. Single use-case problem, there is 1 dataset, 1 model, and 1 prediction.
  2. Using classic models like logistic regression or random forest.
  3. Want to reach the minimum viable product stage quickly.

Stand on the shoulders of greats

  • Model Inventory: PyTorch, HuggingFace, TensorFlow.
  • Model Training and Evaluation: PyTorch Lightning, FastAI
  • Data Versioning: DVC
  • Dependency Management: Conda, pip, poem
  • Experiment Tracking: MLFlow, KubeFlow
  • Model Registry: MLFlow
A lot of toys out there to play with. (Source)

Do not go gentle into the good night

A diagram to help build the large applications. (Author)
  1. Do not hard code any configuration, pass it as an argument or environment variable.
  2. Define your own training and prediction loops to keep control.
  3. Define a callback mechanism that allows you to control training and logging.
  4. Log everything, and build an extensive logging system with proper formatting. Debugging can become hell otherwise.
  5. Build extensive exception handling and implement tests for them. You can define very very small datasets to only be used to test code modules related to training or tuning to avoid long test times.
  6. Build pipelines to bring models, data, and training together for each use case.




Machine Learning Engineer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store