2019 has been a busy year. I joined the Fast Forward Labs team as a Research Engineer focused on Machine Learning. This has proved to be the incredible learning and self development exercise I hoped it would be (thanks to my amazing colleagues). I also got to serve as a Google Developer Expert for Machine Learning. Overall, I had the opportunity to create a few useful things and also give a few talks --- experiences for which I am thankful.

This year, I seem to have committed about 3 times more code than last year. Note: While the graphic above looks at committed code, it is important to mention that it is a limited metric for work done. First, commits may be simply readme edits, one liners etc. Second, much of useful work is not tracked by commits --- thinking, architecting, notebooks/experiments, confidential internal/client projects etc. Committed code however is useful for comparison (this year vs last year) and as a high level measure of activity.

Technical Contributions

There are 3 public projects I am particularly proud of.

Handtrack.js

Handtrack.js is a library for prototyping realtime hand detection (bounding box), directly in the browser. Handtrack.js was a good experience --- creating something for the community and watching it spark excitement among developers (1.5k+ likes on Github and counting).

At some point, Handtrack.js was listed as one of the top 10 most popular Javascript repos on Github. It was #2 for the Month of March.

ConvNet Playground.

Convnet Playground is a tool for the interactive exploration of Convolutional Neural Networks (Convnets or CNNs) applied to the task of semantic search.

It provides 3 datasets of varied complexity and allows the user to explore and compare semantic search performance while using 8 modern pretrained models as feature extractors. For each model, ConvNet Playground also allows the user explore the performance of 8 submodels constructed at each layer with implications for latency.

Anomagram

Anomagram is an interactive visualization of how autoencoders can be applied to the task of anomaly detection. I created it as both a learning tool, and a prototype example of what an ML product interface could look like (of course what I cover is a small slice of the entire process). You can build, train, evaluate an autoencoder ... all in the browser, built with Tensorflow.js! I literally just finished working on this today December 30th!

Train a Model Module: Anomagram provides a direct manipulation interface that allows the user to specify a model (add/remove layers and units within layers), modify model parameters (training steps, batchsize, learning rate, regularizer, optimizer, etc), modify training/test data parameters (data size, data composition), train the model, and evaluate model performance (visualization of accuracy, precision, recall, false positive, false negative, ROC etc metrics) as each parameter is changed. The task is anomaly detection within ECG signal data.

I hope to make these projects better as time permits and excited to see it used by others!

Conferences Talks

I gave a few talks this year.

• NVIDIA GTC: Data2Vis: Automatic Generation of Data Visualizations Using Sequence-to-Sequence Recurrent Neural Networks.
• Strata NY: Handtrack.js: Building gesture-based interactions in the browser using TensorFlow
• Tensorflow World: Handtrack.js: Building gesture-based interactions in the browser using TensorFlow
• QCon: ML in the Browser: Interactive Experiences with Tensorflow.js
• Google Developer ML Summit --- Cambridge and Pittsburgh (video). Art + AI: Generating Novel African Mask Art using Generative Adversarial Networks.

I also got to talk about ML at a few devfest events (Albany, Rochester) and share insights with this brilliant tech community! I did a podcast for QCon and Practal AI.

Writing

I spent some time writing, mostly in the form of blog posts. I wrote about ConvNet Playground, Anomagram, Handtrack.js, ML in the browser. I started writing a couple papers .. hopefully I get to complete them sometime.

The Most Important Thing.

Alot happened this year. But the highlight of the year has been welcoming our little one and spending time with him over the last few months. As I look towards 2020, I look forward to spending more time with family, getting better at ML and finding more opportunities to contribute to the community. Maybe some more academic writing .. maybe.

Cheers to the new year!