MLDev. Experiment reproducibly — Open Data Science

Overview

MLDev facilitates running data science experiments, helps in results presentation and eases paper preparation for students, data scientists and researchers.

The MLDev software provides the following features to help with automating machine learning experiments:

Configure stages and parameters of a data science experiment separately from your code
Conduct a repeatable experiment locally, on Google Colab or PaperSpace
Keep versions of your code, results and intermediate files on Google Drive (other repos coming soon)
Use a variety of pre-built templates to get started: see template-default and template-intelligent-systems

MLDev also provides experiment services that run alongside your pipeline:

You can have the notifications via Telegram about the exeptions while training your model
Keep updated on current experiment state using TensorBoard (even on Colab)
Deploy and demo your model with a model controller (feature in progress)

Installation

MLDev is currently in the Development stage and supports Ubuntu 16.04+.

Get the latest version of the install script to your local machine and run it.

$ curl https://gitlab.com/mlrep/mldev/-/raw/develop/install_mldev.sh -o install_mldev.sh
$ chmod +x ./install_mldev.sh
$ ./install_mldev.sh base

You may be asked for root privileges if there are system packages to be installed.

Wait a couple of minutes until installation is done. You are almost ready to use our instrument, congrats!

Then get the default demo experiment

$ mldev init <new_folder>

Answer the questions the setup wizard asks or skip where possible.

Then run the default pipeline of the experiment

$ cd <new_folder>
$ mldev run

How to use

See more at the project Wiki

Examples

Here are several examples where MLDev has been applied

https://github.com/Intelligent-Systems-Phystech/2021-Project-74
https://github.com/prog-autom/hidden-demo
https://gitlab.com/prog-autom/ganrl_nps_solution/-/tree/mldev

More to come…

License

The source code of the project is licensed under Apache 2.0 license.

How to contribute

MLDev is a community open source project. Everyone is welcome to participate!

For python developers and ML enthusiasts we have tasks on experiment management and visualizations.

Data scientists may find it interesting to contribute model analysis and AutoML tool integrations.

You may also wish to get your MLDev experiment project listed on the project web page or contribute a reproduction of a research paper.

We also welcome data science experts and educators for collaboration on teaching materials and advice on the use of MLDev in the teaching process (like data analysis lab or homeworks).

Please, see also CONTRIBUTING.md and developer docs.