#35: Completely automatic video background removal with unscreen, and circuits for understanding neural networks

Hey everyone, welcome to Dynamically Typed #35.

To get the most important thing out of the way first: the novel coronavirus has spread significantly in the past month, and the Netherlands—where I live—has just now (an hour ago as of writing) announced a major shut down of public life. Besides government reports and accredited news sites, here are two resources I’ve found useful in understanding this pandemic:

• Oxford University’s excellent Our World in Data visualizations of the most recent World Health Organization (WHO) data surrounding the virus (especially this table)
• The Washington Post’s explainer on the importance of social distancing

Please make sure you’re self-isolating as much as possible. Stay safe.

Writing today’s issue has helped me (somewhat) refrain from constantly refreshing the news, so I hope it finds its way into your inbox as a happy distraction too. I didn’t find a lot of new climate change AI updates or machine learning art over the past two weeks, so today’s edition of DT is a bit shorter than usual.

Productized Artificial Intelligence 🔌

Landing page for unscreen

Unscreen is a new zero-click tool for automatically removing the background from videos. It’s the next project from Kaleido, the company behind remove.bg, which I’ve covered extensively on Dynamically Typed: from their initial free launch (DT #3) and Golden Kitty award (DT #5), to the launch of their paid photoshop plugin (DT #12) and cat support (yes, really: DT #16).

Unscreen is another great example of a highly-targeted, easy-to-use AI product, and I’m excited to see it evolve—probably following a similar path to remove.bg, since they’ve already pre-announced their HD, watermark-free pro plan on the launch site.

Quick productized AI links 🔌

• 🚘 Self-driving car company Waymo has raised a big new round of funding.
• 🏋️‍♀️ Natt Garun at The Verge: Tempo is a smart home gym that uses computer vision to track your form in real time.

Machine Learning Research 🎛

“By studying the connections between neurons, we can find meaningful algorithms in the weights of neural networks."

Chris Olah et al. wrote a fascinating new Distill article about “circuits” in convolutional neural networks. The authors aim to reposition the field of AI interpretability as a natural science, like biology and chemistry:

There are two common proposals for dealing with this [lack of shared evaluation measures in the field of interpretability], drawing on the standards of adjacent fields. Some researchers, especially those with a deep learning background, want an “interpretability benchmark” which can evaluate how effective an interpretability method is. Other researchers with an HCI background may wish to evaluate interpretability methods through user studies.

But interpretability could also borrow from a third paradigm: natural science. In this view, neural networks are an object of empirical investigation, perhaps similar to an organism in biology. Such work would try to make empirical claims about a given network, which could be held to the standard of falsifiability.

Olah et al. do exactly this by investigating the Inception v1 network architecture in detail and presenting three speculative claims about how convolutional neural networks work:

1. Features are the fundamental unit of neural networks. They correspond to directions. These features can be rigorously studied and understood.
2. Features are connected by weights, forming circuits. These circuits can also be rigorously studied and understood.
3. Analogous features and circuits form across models and tasks.

For the former two claims, they present substantive evidence: examples of curve detectors, high-low frequency detectors, and pose-invariant dog head detectors for their claim about features; and examples of again curve detectors, oriented dog head detection, and car + dog superposition neurons for the circuits claim.

As always, the article is accompanied by very informative illustrations, and even some interesting tie-backs to the historical invention of microscopes and discovery of cells. I found it a fascinating read, and it made me think about how these findings would look in the context of binarized neural networks. You can read the article by Olah et al. (2020) on Distill: Zoom In: An Introduction to Circuits.

An example of a local narrative from Google’s Open Images V6 dataset.

Quick ML research + resource links 🎛 (see all 57 resources)

• 🖼 Google released Open Images V6, a new version of “the largest annotated image dataset in many regards.” It now features local narratives, such as the one I embedded above, consisting of “synchronized voice, text, and mouse traces over the objects being described.”
• ⚡️ HiPlot is Facebook Research’s new “lightweight interactive visualization tool to help AI researchers discover correlations and patterns in high-dimensional data using parallel plots and other graphical ways to represent information.”
• ⚡️ Neural Tangents is a high-level neural network API for specifying complex, hierarchical, neural networks of both finite and infinite width.

Thanks for reading! As usual, you can let me know what you thought of today’s issue using the buttons below or by replying to this email. If you’re new here, check out the Dynamically Typed archives or subscribe below to get a new issues in your inbox every second Sunday.

If you enjoyed this issue of Dynamically Typed, why not forward it to a friend? It’s by far the best thing you can do to help me grow this newsletter. ☔️