#9: OpenAI and Google's activation atlases, a16z's ML startup investments, and Microsoft's AI pipeline

It’s been another busy two weeks in the world of productized artificial intelligence and machine learning technology. Today, I’m covering a big list of a16z’s ML startup investments; Google AI and OpenAI’s collaboration on an exciting new neural network visualization tool called activation atlases; Microsoft’s AI pipeline from research to products; some cool new AI art; and much more.

I’m also experimenting with a new quick resource links list as part of the ML tech section, to cover useful new open-source libraries and resources without having them take up too much newsletter space or writing time—let me know what you think about it!

Productized Artificial Intelligence 🔌

The ways in which AI and humans will work together for a better society, according to Frank Chen. (Andreessen Horowitz)

Frank Chen of Silicon Valley venture capital firm Andreessen Horowitz (a16z) wrote about how AI and humans can work together in a blog post version of his talk at the November 2018 a16z Summit. He introduces lots of AI-powered startups (many of which are in the a16z portfolio) that enable this human-machine collaboration across five categories. Here’s a rundown:

1. Automating the routine enables us to be more creative:People.ai automatically populates data into Customer Relationship Management (CRM) software for marketing and sales companies; Everlaw processes and sorts files relevant to a case for legal firms; Dialpad analyzes support calls and extracts pain points; Suki listens to doctor-patient conversations to keep medical files up to date.
2. Machine learning gives us superpowers in the physical world:Pindrop can identify fraud by listening to the audio quality of phone calls; Doxel’s robots scan construction sites to track progress and find potential issues; Tesla’s cars can detect accidents two cars ahead of you and hit the brakes; OrCam’s clip-on camera for glasses can tell people with impaired vision things like how much cash they have in their hand.
3. Helping Us Make Better Decisions:Cresta.ai figures out best practices from sales chats; KoBold is analyzing geological data to find likely locations for cobolt; Branch is using data to give loans to people without credit history.
4. _Automating dangerous jobs & tasks makes us safer: _Zipline uses drones to deliver blood and vaccines in Rwanda; Shield.ai sends drones into buildings for reconnaissance.
5. Machine learning will help us understand each other better:Affectiva and Brain Power help autistic children understand emotional states of other people; Crisis Text Line uses ML to help counselors figure out who is at risk of self-harm.

The big trend in many of these is that AI can take over repetitive or dangerous parts of jobs, leaving humans in charge of the creative side. Earlier this week, EdIntelligence put on a panel discussion on the Future of Work, where the panelists echoed this perspective. Subramanian Ramamoorthy argued that “human creativity will continue to find a balance between what’s automated and what’s not automated,” and journalist Paul Mason said it’s already happening now, with people working four-day weeks without a loss in productivity because of back-business process automation. More here:

• Chen’s full talk: Better Together: Humanity + Machine Learning (YouTube)
• Chen’s blog post: Humanity + AI: Better Together
• My full notes on the panel discussion: Future of Work | We Need to Talk About AI (Notion)

Google has added on-device speech recognition to Gboard, the custom software keyboard its Pixel phones. This is really cool for two reasons. First, it’s based on a character-level recurrent neural network, so it’s eerily fast—“just as if someone was typing out what you say in real-time,“ according to Google. Second, it’s completely on-device and offline, so it doesn’t need wait on sending and receiving data from a remote server. More here:

• Google’s blog post: An All-Neural On-Device Speech Recognizer
• A Tweet by Paul Ford, who has tried the feature out says that it’s “so ridiculously good that [it] feels like uncanny super future magic” (Twitter)

Fast Company did a feature on Microsoft’s pipeline for getting its AI research into products. In the past, Microsoft Research (MSR) was often criticized for “struggling to turn their innovations into features.” Under CEO Satya Nadella, this has changed, with internal conferences and hackathons that put MSR researchers and product teams in the same room to come up with projects. Harry Shum, Microsoft’s executive VP of AI and research, thinks this new collaboration works quite well:

“Nowadays, to do a lot of very exciting AI research, you need to get access to real problems and you need to get access to data,” says Shum. “This is where you work together with [product teams]. You build a new model, you train the new model, and then you tweak your new model. Now you have advanced your basic research further. And along the way, you never know–you could get a breakthrough.”

Read the full feature here, which includes lots of examples of MSR’s AI tech making it into products like Office and Xbox. Harry McCracken at FastCompany: This is Microsoft’s AI pipeline, from research to reality.

You can now take a photo of a spreadsheet and automatically import it into Excel. This is a good example of Microsoft’s AI research making it into products: using your phone, take a photo of a printed-out data table in a newspaper, and the app can automatically turn it into a fully editable Excel file. It even figures out things like units, header rows, and which columns represent sums or averages. Tom Warren at The Verge: Microsoft Excel will now let you snap a picture of a spreadsheet and import it.

Machine Learning Technology 🎛

A layer of an activation atlas for the InceptionV1 network (Carter et al.)

Researchers at Google Brain and OpenAI have released activation atlases, a new Distill paper on visualizing neural networks, building on previous work that visualized individual neurons. An activation atlas is constructed by collecting millions of activation vectors from individual training examples, arranging them in a 2D space so that similar ones are close together, gridding up that space, and then finally visualizing the average activation of each cell.

You can use the atlas to inspect what type of features your network is “looking at” when it makes predictions. For example, the authors constructed an activation atlas for a binary classification problem: differentiating between frying pans and woks. They noticed that the presence of noodles seemed to push the network toward calling something a wok. Indeed, by adding a photo of noodles to the corner of a picture of a frying pan, they could then convince the network that the frying pan was a wok. Similarly, they could fool a classifier into thinking a grey whale was a shark by superimposing a small photo of a baseball.

Carter and his co-authors hope that activation atlases will be used to find more of these kinds of correlations-but-not-causations that networks are picking up and using for classification, to make it easier to find bugs and biases. More here:

• The fully interactive paper on Distill: Exploring Neural Networks with Activation Atlases
• Google’s blog post, featuring examples of different “areas” of the InceptionV1 activation atlas: Exploring Neural Networks with Activation Atlases
• A high-level (non-technical) story on activation atlases from Gregory Barber at Wired: Shark or Baseball? Inside the ‘Black Box’ of a Neural Network
• OpenAI’s blog post: Introducing Activation Atlases

The Gradient, in collaboration with Stanford AI, published a comprehensive ethics guide for machine learning researchers. Specifically, the guide focuses on “what researchers can do to avoid unintended negative consequences of their work.” It’s split up into broad sections with best practices for education, communication & distribution, and advocacy. If you’re an AI student or a ML researcher, I highly recommend you read it here: In Favor of Ethical Best Practices in AI Research.

Quick resource links. Here are a few more links to open-source ML resources that were released in the past two weeks:

• OpenAI’s new multiagent reinforcement learning environment and baseline: Neural MMO: A Massively Multiagent Game Environment
• Google’s new DNN training parallelization library: Introducing GPipe, an Open Source Library for Efficiently Training Large-scale Neural Network Models
• DeepMind’s new library to simplify deploying TensorFlow models on GPUs and Cloud TPUs: TF-Replicator: Distributed Machine Learning for Researchers

Tech and Startups 🚀

OneWeb has launched its first six washing machine-sized internet satellites into orbit. They’re a part of an initial fleet of 650 satellites meant to bring affordable internet to the three billion people who live in areas that currently don’t have wired connectivity. Although OneWeb’s satellites cost less than a 10th to build than traditional internet satellites, it’s not yet clear whether they’ll win this competition for mainstream satellite internet: SpaceX is also doing this, and they launched their first two test satellites about a year ago. Ashlee Vance at Bloomberg: OneWeb Launches First Internet Satellites in Race With SpaceX.

Uber’s electric bikes are getting more popular than their cars. Uber bought electric bike-sharing company Jump last year and rolled it into their service in a few cities; in Sacramento, California, the bikes are now used more than Uber taxis. I expect that this trend will spread as cities in the US start to (slowly) realize that they’re much better off prioritizing public transport, cycling, walking, and micromobility over roads, cars, and parking lots. (See also the micromobility section in DT #6.) Micah Toll at Electrek: Uber’s electric bicycles are starting to get more riders than their cars

Cool Things ✨

Memories of Passersby I by Mario Klingemann (ONKAOS)

This is definitely the coolest AI art I’ve seen so far. Over the past year or two, the AI art community has become quite good at training Generative Adversarial Networks (GANs) on anything from landscapes to fashion shows, and wrangling it to spit out new pictures. Memories of Passersby I by Mario Klingemann is follows this trend, but it is extremely well-executed. Klingemann trained his GAN on a hand-picked collection of 17th, 18th and 19th century portraits; his installation then uses the GAN to continuously generate new faces that morph into each other every few seconds.

While looking at those screens, you would likely see an image that would never exist again. … For Klingemann, finding a way to incorporate this aspect of GANs into his artwork is at least something he’s happy to keep exploring. “I’m not saying I won’t pursue other paths later, but this is closer to showing the potential that these machines have,” he says. To him, the endless parade of portraits in Memories of Passersby I better captures the feeling of unceasing production from AI — “the almost overwhelming output [that] will never stop.”

The installation was auctioned off for 40,000 GBP by Sotheby’s. Check out this article by James Vincent at The Verge, which includes gifs of Memories of Passersby I and some similar AI art: A never-ending stream of AI art goes up for auction.

Thanks for reading! As always, let me know what you thought of this issue using the buttons below or by sending me a message. If you’re new here, subscribe for a new issue of Dynamically Typed every second Sunday!