#31: Uber's Generative Teaching Networks, ML Super Resolution in Pixelmator Pro, and Evolve Energy
Hey everyone, welcome to Dynamically Typed #31! Today’s issue is back to the familiar format, with some productized AI—a new machine learning-powered image upscaling feature in the Pixelmator Pro photo editing app—and some ML research—Uber’s Generative Teaching Networks for faster neural architecture searches. For climate change AI, I’m covering a Texas-based startup called Evolve Energy that uses machine learning to predict the carbon intensity of the electricity grid over time and adjust its users’ energy use accordingly.
Finally, I’m also trying something new: in each section, I’m including a few more cool links that came across my radar, but that I’m not covering in detail. There are quite a few of those in today’s DT, since my mom is visiting me in the Netherlands, and we’re painting my apartment this weekend! Three of my living room walls are now a gorgeous “ficus” dark green. 🍃
Productized Artificial Intelligence 🔌
Traditional Lanczos image scaling (left) vs. Pixelmator’s ML Super Resolution (right).
Photo editing app Pixelmator Pro has added a machine learning-powered image resizing function. There are lots of traditional algorithms for increasing the resolution of an image. For a 2x increase, like going from a 500 x 300 pixel image to 1,o00 x 600 px, for example: nearest neighbor interpolation simply makes four copies of each pixel; bilinear interpolation fills new pixels with the average color of its four neighboring pixels; and Lanczos interpolation is a bit more complex. What these traditional algorithms have in common is that they’re static, not learned: people thought long and hard about them, and then wrote them down as formulae.
But in recent years, much like in other computer vision tasks like image classification and object detection, machine learning has started competing with these traditional algorithms. In a supervised task called superresolution , a convolutional neural network model is trained on pairs of low-resolution and high-resolution images—a dataset that can be generated quite easily by taking HD photos and down-scaling them. From this data, the model can learn to generate high-resolution images from low-resolution inputs. For more technical details, see this survey by Wang, Chen and Hoi (2019).
Pixelmator Pro has now added such a model to its app, in an easy-to-use way that requires no AI knowledge to use: ML Super Resolution is simply another button to click in the app, making this a great example of productized AI. Pixelmator wrote a very good blog post announcing the feature—including details like their network design and use of Apple’s Core ML 3 framework to speed up computation!—which you can read here: All about the new ML Super Resolution feature in Pixelmator Pro.
Quick productized AI links 🔌
- 📱 Follow-up: Itay Inbar and Nir Shemy for the Google AI Blog: The On-Device Machine Learning Behind Recorder, covering the tech behind the on-device transcription feature I covered in DT #25.
Machine Learning Research 🎛
An overview of generative teaching networks (GTNs).
Such et al. at Uber AI labs introduced Generative Teaching Networks (GTNs), a method to generate synthetic training data to speed up neural architecture search (NAS) algorithms. A NAS is a way to explore a design space of potential neural network architectures with different layer counts, depths, connections, etc. to find an architecture that performs well on a given task and set of design constraints. A NAS may explore a space of many thousands of possible networks, and each network needs to be trained (to some extent) to evaluate its performance on the task at hand. So neural architecture searches are usually computationally very expensive to perform, and have historically mostly been done by well-funded corporate AI labs; see MobileNetV3 by Howard et al. (2019) at Google AI and FBNet by Wu et al. (2019) at Facebook AI Research for example.
Here’s how it works, following the diagram above:
A GTN is like the generator in a generative adversarial network (GAN), except without a pressure to make data look realistic. Instead, it produces completely artificial data that a never-seen-before learner neural network (with a randomly sampled architecture and weight initialization) trains on for a small number of learning steps, e.g., the steps could be via stochastic gradient descent (SGD). Afterwards, the learner network–which so far has never seen real data–is evaluated on real data (e.g., whether it can recognize handwritten images in the classic MNIST dataset), which provides the meta-loss objective that is being optimized. We then differentiate through the entire learning process via meta-gradients to update the GTN parameters to improve performance on the target task. The learner is then discarded and the process repeats.
By training for fewer steps on synthetic GTN-generated data, Such et al.’s NAS sped up 9x! “GTN-generated data is thus a faster drop-in replacement for real data in NAS algorithms.” This is very cool, and GTN-generated datasets may open neural architecture searches up to more research teams. Read more on the Uber engineering blog or in the paper by Such et al. (2019).
Related, also check out this Toward Data Science post by Jesus Rodriguez: Uber Has Been Quietly Assembling One of the Most Impressive Open Source Deep Learning Stacks in the Market. I’ve covered some tools of the tools Rodriguez mentions here in the newsletter, like Michelangelo (DT #5) and Ludwig (DT #7).
Quick ML research links 🎛
- 👨🔬 Jeff Dean wrote a detailed post about Google’s past year of AI research and their future plans: Google Research: Looking Back at 2019, and Forward to 2020 and Beyond. Climate AI is mentioned less than I expected.
- 🧮 Cool new results form Charton and Lampe at Facebook AI research: Using neural networks to solve advanced mathematics equations. They tackle the problem using neural machine translation techniques, and even manage to solve some equations that advanced specialized software like Mathematica and Matlab can’t solve. (Thanks Casper!)
- 👁 Jeremy Howard’s fast.ai introduction to self-supervised learning for vision: Self-supervised learning and computer vision
- 🐦 New Distill publication by Strumfels, Lundberg and Lee: Visualizing the Impact of Feature Attribution Baselines
Quick ML resource links ⚡️ (see all 50)
- Jeremy Howard’s (fast.ai) data project checklist, covering strategy, data, analytics, implementation, maintenance, and constraints considerations.
Artificial Intelligence for the Climate Crisis 🌍
Evolve Energy uses machine learning to predict the carbon intensity of the energy grid. Through its app, it then automatically adjusts its users’ energy use to be higher during cheaper and lower-impact times. For example, when Evolve predicts a windy, sunny morning but a cloudy, calm afternoon, it makes sure you charge your electric car in the morning when there’s lots of solar and wind energy available. Seth Colaner for VentureBeat:
The technology includes pulling in real-time data, like how and when customers are using electricity, ambient temperatures, time of day, duration, patterns in preferences and behaviors, and so on. … Based on that data, Evolve Energy runs a recommendation engine that tells customers what it’s going to do for them. Customers can override the recommendations if they choose, and the app will instantly inform them what the resultant costs will be. It’s a balance, Lee said, involving transparency, control, and feedback.
Evolve can do all this because, instead of selling customers power at a flat rate like other power companies do, it resells power at wholesale prices, in 5-minute increments without charging a markup. Instead, it makes money by charging a $10/month fee for its app and predictive technology. CEO Michael Lee: “And by doing that, we are now aligning our interests for our customers to no longer sell them … electricity, but to become an advisor to our customers.” I love this alignment of incentives, and it’ll be exciting to see products like Evolve spread to other regions. (They’re currently only in Texas.) Check out Evolve Energy’s website for more details about how it works and how it helps decrease carbon emissions.
(Related: WattTime is a nonprofit alternative, though it’s slightly less consumer-facing.)
Quick climate AI links 🌍
- 🌧 Jason Hicket for the Google AI Blog: Using Machine Learning to “Nowcast” Precipitation in High Resolution
- 🛢 David on the Climate Change AI forum: Artificial Intelligence and the Fossil Fuel Industry, covering the Vox video that recently made the rounds recently (Google and Amazon are now in the oil business) and more.
Thanks for reading! As usual, you can let me know what you thought of today’s issue using the buttons below or by replying to this email. If you’re new here, check out the Dynamically Typed archives or subscribe below to get a new issues in your inbox every second Sunday.
If you enjoyed this issue of Dynamically Typed, why not forward it to a friend? It’s by far the best thing you can do to help me grow this newsletter. 🤟