#19: Microsoft's $1B OpenAI investment, Lyft's dataset, and what makes a peacock a peacock?

Hey everyone, welcome to Dynamically Typed #19. Today’s edition is a bit shorter than usual, but with good reason: I’m writing it early because the rest of this week I’ll be super busy with Amsterdam Pride events! 🏳️‍🌈🎉

Anyway: the past fortnight’s biggest news was OpenAI’s $1 billion investment in OpenAI and the community’s reaction to their partnership. Other stories include Ben Evan’s blog post about the productization of computer vision on the edge, Misa Ogura’s FlashTorch feature visualization toolkit, and Lyft’s open-sourcing of a self-driving car dataset. Finally, National Geographic covered some interesting work being done in the AI + climate change space. Let’s dive in.

Productized Artificial Intelligence 🔌

OpenAI + Microsoft

Microsoft is investing $1 billion in OpenAI.

To raise enough money to pay for the recruiting and computing needed to compete with the likes of DeepMind, much of the OpenAI nonprofit (and its employees) has been pivoted to become the for-profit OpenAI LP. The community’s reaction, unsurprisingly, has been mixed; especially OpenAI’s lofty marketing talk about the pre- and post-artificial general intelligence (AGI) periods of the company’s future raised some eyebrows.

This “pre-AGI period” is also where the Microsoft investment and partnership comes in: they will become OpenAI’s “exclusive cloud provider” and their main strategy for productizing the lab’s AI research.

[We’ll] be working hard together to further extend Microsoft Azure’s capabilities in large-scale AI systems. … [We] intend to license some of our pre-AGI technologies, with Microsoft becoming our preferred partner for commercializing them.

This is great for Microsoft’s AI cloud (and klout), but I’m not too sure what it says about the future of OpenAI. Since they won’t disclose the terms of the investment, it’s hard to know how much the company’s original mission—“the creation of beneficial AGI”—will fall to the wayside to make room for productization.

(And if it does, is that necessarily a bad thing? After all, productized AI is what this section of Dynamically Typed is all about. I’d love to hear your thoughts, so use that reply button!)

Read more about OpenAI + Microsoft here:

• OpenAI press release: Microsoft Invests In and Partners with OpenAI to Support Us Building Beneficial AGI
• Tom Simonite for Wired: To Compete With Google, OpenAI Seeks Investors—and Profits

Ben Evans of Andreessen Horowitz wrote a post about the potential of computer vision to touch almost everything. On the back of imaging sensors that have become ridiculously cheap in recent years (because of the efficiency of the smartphone supply chain), Evans argues that “imaging plus ML” will power a lot more AI computing on the edge:

The common thread across all of this is that vision replaces single-purpose inputs, and human mechanical Turks, with a general-purpose input.

I think that especially the former makes for an interesting thought experiment. Let’s take a look at fire detection: in the past, we found out a house was on fire only once a human saw it; then, smoke alarms were invented that started doing that job (and saving lives) automatically; now, imaging plus ML can “see” a fire on a camera feed—without the need of a single-purpose input. Of course, smoke alarms are already cheap and widely available, so they probably won’t get replaced by AI cameras.

But what other specialized sensors can become more affordable or more ubiquitous if they can be replaced by a computer vision model? And what things that a human can monitor now, for which we have no specialized sensors, can we start to track using AI-powered cameras? That’s where cheap imaging plus ML will have a huge impact. (Pervasive facial recognition and image censorship, sadly, are obvious immoral examples that are already being put into production.)

As always, Evan’s post is worth a read: Computers that can see.

Machine Learning Research 🎛

_FlashTorch visualizations of parts of images that have the most influence on _

Misa Ogura wrote about FlashTorch, her open source feature visualization toolkit for neural networks in PyTorch. Her post first gives a nice history of feature visualization:

Feature visualisation is an active area of research which aims to understand how neural networks perceive images, by exploring ways in which we can look “through their eyes”. It has emerged and evolved in response to an increasing desire to make neural networks more interpretable to humans.

Ogura then covers class saliency maps, activation maximization, neuron arithmetic, and activation atlases (see DT #9). Finally, she introduces FlashTorch, which implements saliency maps to show what parts of an image has the most influence on a classification, and dives into some some of its implementation details. As someone who isn’t too deep into computer vision (yet 🤫), I learned a lot from this post. More:

• Misa Ogura for Towards Data Science: Uncovering what neural nets “see” with FlashTorch
• GitHub repository: MisaOgura/flashtorch
• Colab notebook: Visualise image-specific class saliency with backpropagation

Lyft has open-sourced its autonomous driving dataset. It includes 55,000 human-labeled frames of 3D imagery, each taken from 7 cameras and 3 lidars:

We’re thrilled to share a comprehensive, large-scale dataset featuring the raw sensor camera and LiDAR inputs as perceived by a fleet of multiple, high-end, autonomous vehicles in a bounded geographic area. This dataset also includes high quality, human-labelled 3D bounding boxes of traffic agents, an underlying HD spatial semantic map.

Data collection and labeling like this is immensely expensive, so I’m happily surprised to see Lyft take such a collaborative approach to the self-driving car problem. I wonder how they convinced their investors to OK this. Either way, check out Lyft’s page for the dataset on their Level 5 site, which includes some very cool interactive dataset samples: Lyft Level 5 Dataset: Moving autonomous vehicles forward, together.

Quick ML resource links ⚡️ (see all 32)

• SHapley Additive exPlanations (SHAP) is a unified approach to explain the output of any machine learning model using seven different methods. Link: slundberg/shap

Artificial Intelligence for the Climate Crisis 🌍

National Geographic interviewed David Rolnick , co-author of “Tackling Climate Change with Machine Learning,” the 97-page vision paper that came out of the similarly named ICML workshop earlier this year (see DT #16).

“It’s surprising how many problems machine learning can meaningfully contribute to,” says Rolnick, who also helped organize the June workshop.

The paper offers up 13 areas where machine learning can be deployed, including energy production, CO2 removal, education, solar geo-engineering, and finance. Within these fields, the possibilities include more energy-efficient buildings, creating new low-carbon materials, better monitoring of deforestation, and greener transportation. However, despite the potential, Rolnick points out that this is early days and AI can’t solve everything.

“AI is not a silver bullet,” he says.

The article then goes deeper into three specific areas where machine learning researchers can help in mitigating the climate crisis:

• Better climate predictions, including an ensemble learning method being developed by University of Colorado professor Claire Monteleoni, who co-founded the Climate Informatics workshop series (in 2011!) and presented about climate change + AI at NeurIPS (in 2014!).
• Showing the effects of extreme weather , including work by Victor Schmidt who used GANs to visualize what effect extreme weather (intensified by climate change) can have on peoples’ homes (see DT #14).
• Measuring where carbon is coming from, including Carbon Tracker, which uses computer vision to analyze where polution comes from using satellite photos.

For in-depth views into each of these projects, check out Jackie Snow’s article for National Geographic: How artificial intelligence can tackle climate change.

Thanks for reading! As usual, you can let me know what you thought of today’s issue using the buttons below or by replying to this email. If you’re new here, check out the Dynamically Typed archives or subscribe below to get a new issues in your inbox every second Sunday.

If you enjoyed this issue of Dynamically Typed, why not forward it to a friend? It’s by far the best thing you can do to help me grow this newsletter. 🏳️‍🌈