#25: AI-powered rainforest monitoring, Google's Pixel 4, and OpenAI's Rubik's cube-solving robot hand

Hey everyone, welcome to the 25th edition of Dynamically Typed! This week, I’m covering the launch of Google’s new Pixel 4 phone for productized AI; and OpenAI’s Rubik’s cube-solving robot hand and an analysis of machine learning frameworks for ML research. On the climate AI side, I found two very cool new projects: one that uses old phones to monitor illegal logging in rainforests, and one that uses satellite imagery to monitor forest health. Let’s jump in.

Productized Artificial Intelligence 🔌

Pikachu teaches users how to use the Pixel 4’s new Motion Sense features. (The Verge)

Google’s Pixel 4 launched with a bunch of new AI-powered features. The biggest two are Motion Sense and on-device ML:

• Motion Sense is powered by a front-facing radar chip that can recognize hand gestures. This means the phone can, for example, turn its screen on (or lower the volume of your morning alarm!) when it senses that you’re reaching for it, before you even touch it. Although you can also use gestures to control audio playback, Google says it has plans to add a lot more functionality to Motion Sense in the future.
• On-device transcription is another super cool feature. Google has embedded an entire model for recognizing English-language on its Neural Core chip, enabling things like captioning any audio playing anywhere in the system (like a podcast or your own videos) and a voice memos app that instantly transcribes the audio it’s recording. Check out this demo by Andy Ihnatko on Twitter.

Besides cameras, I think that these kinds of on-device machine learning applications will become one of the biggest aspects that phone makers (especially Google and Apple) will be competing on over the next few years. Read all about the Pixel 4 in Dieter Bohn’s piece for The Verge: Google Pixel 4 and 4 XL hands-on.

Bonus: Ben Thompson also had a good Stratechery take on how the Pixel 4 fits into Google’s larger, renewed company vision: Google and Ambient Computing.

Machine Learning Research 🎛

The robot hand is robust to several types of perturbation, including being hit by a plush giraffe. (OpenAI))

OpenAI used reinforcement learning to train a robot hand to solve a Rubik’s cube. This is very cool because complex object manipulation tasks like this have been a challenge in the robotics industry for decades. OpenAI’s main contribution here is an Automatic Domain Randomization (ADR) method that makes it possible to transfer their simulated models—which solved robotic Rubik’s cube manipulation in 2017—to the real world:

ADR starts with a single, nonrandomized environment, wherein a neural network learns to solve Rubik’s Cube. As the neural network gets better at the task and reaches a performance threshold, the amount of domain randomization is increased automatically. This makes the task harder, since the neural network must now learn to generalize to more randomized environments. The network keeps learning until it again exceeds the performance threshold, when more randomization kicks in, and the process is repeated.

The actual solving (deciding which turns need to be made) is done by an existing algorithm that was not learned, and the position of the cube isn’t analyzed visually but using built-in sensors. This has led to some controversy on Twitter, but I think that’s a bit overblown: the scope of the project—object manipulation—was pretty clear to me from the blog post. More here:

• OpenAI blog post: Solving a Rubik’s Cube with a Robot Hand
• Paper on arXiv: Solving Rubik’s Cube with a Robot Hand

Horace He analyzed the 2019 state of machine learning frameworks in research and production, following the releases of TensorFlow 2.0 and PyTorch 1.3 earlier this month (see DT #24). He scraped papers from major AI conferences and counted how many mentioned the two frameworks by name. His conclusion is that while TensorFlow remains very popular in industry, PyTorch is increasingly dominating on the research side:

In 2018, PyTorch was a minority. Now, it is an overwhelming majority, with 69% of CVPR using PyTorch, 75+% of both NAACL and ACL, and 50+% of ICLR and ICML. While PyTorch’s dominance is strongest at vision and language conferences (outnumbering TensorFlow by 2:1 and 3:1 respectively), PyTorch is also more popular than TensorFlow at general machine learning conferences like ICLR and ICML.

He also thinks that TensorFlow’s industry position may be weakened by PyTorch’s popularity amongst researchers and graduating PhD students, and its recent focus on deployability. The full analysis is an interesting read, but definitely to be taken with a grain of salt: He interned at the PyTorch JIT team this summer, and the study did not consider Keras, an important part of modern TensorFlow. Read it at The Gradient: The State of Machine Learning Frameworks in 2019.

Artificial Intelligence for the Climate Crisis 🌍

One of 20tree.ai’s services is monitoring urban trees to help keep cities sustainable in a changing climate.

20tree.ai is using machine learning to monitor forests from satellite imagery. The startup grew out of a Kaggle competition about “tracking the human footprint in the Amazon rainforest” hosted by satellite imagery company Planet. It now offers a range of services including monitoring tree health and threats (droughts, insect plagues, storm damage, etc.), tree features and productivity (species, height, diameter, growth predictions, etc.), and soil and water quality. They can even detect early signs of illegal deforestation operations, as Aniek Schouten explains on the 20tree.ai blog:

Deforestation is associated with specific patterns that appear in forests. When these patterns appear in a forest, it tells us that the chance is high that large scale deforestation is about to happen. For example little roads appear in a previously untouched forest. We see narrow roads and winding trails reaching into a dense forest area, leading towards a river or a road. These roads are often a sign of unsustainable selective logging.

Protecting and sustaining the world’s forests will be an important aspect of our fight against the climate crisis, so seeing conservation-focused startups like 20tree.ai pop up is very encouraging. Read more about 20tree.ai’s work on their blog:

• Aniek Schouten: How can machine learning help to feed 9.8 billion people in 2050?
• Indra den Bakker: From Kaggle competition to start-up and tracking 2 million km² of forest

“Topher White installing a solar-powered listening unit in a rainforest on the Indonesian island of Sumatra in July.” (Ulet Ifansasti for The New York Times)

Conservationists in Indonesia are using old phones and AI to monitor illegal logging. The monitoring system was installed by California-based nonprofit Rainforest Connection, which builds devices powered by solar panels and used phones and installs them in treetops in forests around the world. The phones have powerful microphones and continuously upload an audio feed to the cloud:

It is analyzed in real time by artificial-intelligence software capable of distinguishing the sounds of chainsaws, logging trucks and other telltale audio signatures of illegal activity. The software then sends rangers instant alerts, through a specialized app that, in theory, could help them make arrests.

The devices have a range of about a mile, and they have a nice secondary purpose of providing data for scientists monitoring populations of endangered bird species. But they also illustrate another aspect of AI approaches to climate problems: they’re just one small part of a solution. If there is bad cellular data coverage, or illegal loggers aren’t prosecuted, there’s not much the monitoring software can do. Read Mike Ives’ full story (with some more amazing photos!) for the New York Times here: Using Old Cellphones to Listen for Illegal Loggers.

Thanks for reading! As usual, you can let me know what you thought of today’s issue using the buttons below or by replying to this email. If you’re new here, check out the Dynamically Typed archives or subscribe below to get a new issues in your inbox every second Sunday.

If you enjoyed this issue of Dynamically Typed, why not forward it to a friend? It’s by far the best thing you can do to help me grow this newsletter. 🚂