There’s a new version of Wordle called Semantle — not one that I “predicted” — that wants you to find the target word by looking not for a chain of spellings but a chain of semantics. For example, if you started with the word “child” you might get to the answer as follows:

1. Child
2. Play
3. game
4. Chess
5. Square
6. Circle
7. Donut
8. Homer

In short, you’re playing word associations except the associations can be very loose. It’s not like Twenty Questions where, once you get down a track (say “animals”), you’re narrowing the scope until there’s only one thing left. In Semantle, the associations can take a sudden turn in any of a thousand directions at any moment.

Which means it’s basically impossible to win.

It is, however, a good introduction to how machine learning “thinks” about words. Or at least one of the ways. Semantle is based on word2vec, which creates text embeddings derived from an analysis of some large — sometimes very very large — set of texts. Text embeddings map the statistical relationships among words based on their proximities in those texts.

In a typical example, word2vec may well figure out that “queen” and “king” are semantically close, which also might well let it figure out that “king” is to “prince” as “queen” is to “princess.”

But there are, of course, many ways that words can be related — different axes of similarity, different dimensions. Those are called “vectors” (as in “word2vec“). When playing Semantle, you’re looking for the vectors in which a word might be embedded. There are many, many of those, some stronger than others. For example, “king” and “queen” share a dimension, but so do “king” and “chess”, “king” and “bed size”, and “king” and “elvis.” Words branch off in many more ways than in Wordle.

For example, in my first game of Semantle, after 45 attempts to find a word that is even a little bit close to the answer, I found that “city” is vaguely related to it. But now I have to guess at the vector “city” and the target share. The target could be “village”, “busy”, “taxi”, “diverse”, “noisy”, “siege”, or a bazillion other words that tend to appear relatively close to “city” but that are related in different ways.

In fact, I did not stumble across the relevant vector. The answer was “newspaper.”

I think Semantle would be more fun if they started you with a word that was at some reasonable distance from the answer, rather than making you guess what a reasonable starting word might be. Otherwise, you can spend a long time — 45 tries to get “city” — just generating random words. But if we knew a starting word was, say, “foot”, we could start thinking of vectors that that word is on: measure, toe, body, shoe, soccer, etc. That might be fun, and would stretch our minds.

As it is, Semantle is a game the unplayability of which teaches us an important lesson.

And now I shall wait to hear from the many people who are actually able to solve Semantles. I hate you all with a white hot and completely unreasonable passion.[1]

[1] I’ve heard from people who are solving it. I no longer hate them.

Be the first to comment »

Notes for a post:

Plato said (Phaedrus, 265e) that we should “carve nature at its joints,” which assumes of course that nature has joints, i.e., that it comes divided in natural and (for the Greeks) rational ways. (“Rational” here means something like in ways that we can discover, and that divide up the things neatly, without overlap.)

For Aristotle, at least in the natural world those joints consist of the categories that make a thing what it is, and that make things knowable as those things.

To know a thing was to see how it’s different from other things, particularly (as per Aristotle) from other things that they share important similarities with: humans are the rational animals because we share essential properties with other animals, but are different from them in our rationality.

The overall order of the universe was knowable and formed a hierarchy (e.g. beings -> animals -> vertebrates -> upright -> rational) that makes the differences essential. It’s also quite efficient since anything clustered under a concept, no matter how many levels down, inherits the properties of the higher level concepts.

We no longer believe that there is a perfect, economical order of things. “We no longer believe that there is a single, perfect, economical order of things. ”We want to be able to categorize under many categories, to draw as many similarities and differences as we need for our current project. We see this in our general preference for search over browsing through hierarchies, the continued use of tags as a way of cutting across categories, and in the rise of knowledge graphs and high-dimensional language models that connect everything every way they can even if the connections are very weak.

Why do we care about weak connections? 1. Because they are still connections. 2. The Internet’s economy of abundance has disinclined us to throw out any information. 3. Our new technologies (esp. machine learning) can make hay (and sometimes errors) out of rich combinations of connections including those that are weak.

If Plato believed that to understand the world we need to divide it properly — carve it at its joints — knowledge graphs and machine learning assume that knowledge consists of joining things as many different ways as we can.

3 Comments »

Aeon.co has posted an article I worked on for a couple of years. It’s only 2,200 words, but they were hard words to find because the ideas were, and are, hard for me. I have little sense of whether I got either the words or the ideas right.

The article argues, roughly, that the sorts of generalizations that machine learning models embody are very different from the sort of generalizations the West has taken as the truths that matter. ML’s generalizations often are tied to far more specific configurations of data and thus are often not understandable by us, and often cannot be applied to particular cases except by running the ML model.

This may be leading us to locate the really real not in the eternal (as the West has traditional done) but at least as much in the fleeting patterns of dust that result from everything affecting everything else all the time and everywhere.

Three notes:

1. Nigel Warburton, the philosophy editor at Aeon, was very helpful, as was Timo Hannay in talking through the ideas, and at about a dozen other people who read drafts. None of them agreed entirely with the article.

2. Aeon for some reason deleted a crucial footnote that said that my views do not necessarily represent the views of Google, while keeping the fact that I am a part time, temporary writer-in-residence there. To be clear: My reviews do not necessarily represent Google’s.

3. My original first title for it was “Dust Rising”, but then it became “Trains, Car Wrecks, and Machine Learning’s Ontology” which i still like although I admit it that “ontology” may not be as big a draw as I think it is.

Be the first to comment »

You’ve probably heard about MyHeritage.com‘s DeepNostalgia service that animates photos of faces. I’ve just posted at Psychology Today about the new type of uncanniness it induces, even though the animations of the individual photos I think pretty well escape The uncanny Value.

Here’s a sample from the MyHeritage site:

And here’s a thread of artworks and famous photos animated using DeepNostalgia that I reference in my post:

https://t.co/MDFSu3J0H1 has created some sort of animate your old photos application and I’m of course using it to feed my history addiction.
I apologise in advance to all the ancestors I’m about to offend.

Very fake history.

I’m sorry Queenie. pic.twitter.com/2np437yXyt

— Fake History Hunter (@fakehistoryhunt) February 28, 2021

More at Psychology Today …

Be the first to comment »

Google has just launched a new podcast that Yannick Assogba (twitter: @tafsiri) and I put together. Yannick is a software engineer at Google PAIR where I was a writer-in-residence for two years, until mid-June. I am notably not a software engineer. Throughout the course of the nine episodes, Yannick helps me train machine learning models to play Tic Tac Toe and then a much more complex version of it. Then our models fight! (Guess who wins? Never mind.)

This is definitely not a tutorial. We’re focused on getting beneath the metaphors we usually use when talking about machine learning. In so doing, we keep coming back to the many human decisions that have to be made along the way.

So the podcast is for anyone who wants to get a more vivid sense of how ML works and the ways in which human intentions and assumptions shape each and every ML application. The podcast doesn’t require any math or programming skills.

It’s chatty and fun, and full of me getting simple things wrong. And Yannick is a fantastic teacher. I miss seeing him every day :(

All nine episodes are up now. They’re about 25 mins each. You can find them wherever you get your podcasts, so long as it carries ours.

Podcast: https://pair.withgoogle.com/thehardway/

Two-minute teaser:  https://share.transistor.fm/s/6768a641

4 Comments »

I was going through a shelf of books I haven’t visited in a couple of decades and found a book I used in 1986 when I taught Introduction to Computer Science in my last year as a philosophy professor. (It’s a long story.) Ethical Issues in the Use of Computers was a handy anthology, edited by Deborah G. Johnson and John W. Snapper (Wadsworth, 1985).

So what were the ethical issues posed by digital tech back then?

The first obvious point is that back then ethics were ethics: codes of conduct promulgated by professional societies. So, Part I consists of eight essays on “Codes of Conduct for the Computer Professions.” All but two of the articles present the codes for various computing associations. The two stray sheep are “The Quest for a Code of Professional Ethics: An Intellectual and Moral Confusion” (John Ladd) and “What Should Professional Societies do About Ethics?” (Fay H. Sawyier).

Part 2 covers “Issues of Responsibility”, with most of the articles concerning themselves with liability issues. The last article, by James Moor, ventures wider, asking “Are There Decisions Computers Should Not Make?” About midway through, he writes:

“Therefore, the issue is not whether there are some limitations to computer decision-making but how well computer decision making compares with human decision making.” (p. 123)

While saluting artificial intelligence researchers for their enthusiasm, Moor says “…at this time the results of their labors do not establish that computers will one day match or exceed human levels of ability for most kinds of intellectual activities.” Was Moor right? It depends. First define basically everything.

Moor concedes that Hubert Dreyfus’ argument (What Computers Still Can’t Do) that understanding requires a contextual whole has some power, but points to effective expert systems. Overall, he leaves open the question whether computers will ever match or exceed human cognitive abilities.

After talking about how to judge computer decisions, and forcefully raising Joseph Weizenbaum’s objection that computers are alien to human life and thus should not be allowed to make decisions about that life, Moor lays out some guidelines, concluding that we need to be pragmatic about when and how we will let computers make decisions:

“First, what is the nature of the computer’s competency and how has it been demonstrated? Secondly given our basic goals and values why is it better to use a computer decision maker in a particular situation than a human decision maker?”

We are still asking these questions.

Part 3 is on “Privacy and Security.” Four of the seven articles can be considered to be general introductions fo the concept of privacy. Apparently privacy was not as commonly discusssed back then.

Part 4, “Computers and Power,” suddenly becomes more socially aware. It includes an excerpt from Weizenbaum’s Computer Power and Human Reason, as well as articles on “Computers and Social Power” and “Peering into the Poverty Gap.”

Part 5 is about the burning issue of the day: “Software as Property.” One entry is the Third Circuit Court of Appeals finding in Apple vs. Franklin Computer. Franklin’s Ace computer contained operating system code that had been copied from Apple. The Court knew this because in addition to the programs being line-by-line copies, Franklin failed to remove the name of one of the Apple engineers that the engineer had embedded in the program. Franklin acknowledged the copying but argued that operating system code could not be copyrighted.

That seems so long ago, doesn’t it?

Because this post mentions Joseph Weizenbaum, here’s the beginning of a blog post from 2010:

I just came across a 1985 printout of notes I took when I interviewed Prof. Joseph Weizenbaum in his MIT office for an article that I think never got published. (At least Google and I have no memory of it.) I’ve scanned it in; it’s a horrible dot-matrix printout of an unproofed semi-transcript, with some chicken scratches of my own added. I probably tape recorded the thing and then typed it up, for my own use, on my KayPro.

In it, he talks about AI and ethics in terms much more like those we hear today. He was concerned about its use by the military especially for autonomous weapons, and raised issues about the possible misuse of visual recognition systems. Weizenbaum was both of his time and way ahead of it.

Be the first to comment »

Note that this is not late-breaking news. It’s breaking news brought to you late: Android contains a really great Google transcription tool.

Live Transcribe transcribes spoken text in real time. So far, it seems pretty awesome at it. And its machine learning model is loaded on your device, so it works even when you’re offline — convenient and potentially less intrusive privacy-wise. (Only potentially, because Google could upload your text when you connect if it wanted to.)

You can download Live Transcribe from the Play Store, but if you’re like me, it will only give you an option to uninstall it. Oddly, it doesn’t show up in my App drawer. You have to go to your phone’s Settings > Accessibility screen and scroll all the way down to find the Live Transcribe option.

Once you turn it on, you’ll get an icon all the way at the bottom of your screen, to the right of the Home button. Weird that it’s given that much status, but there it is.

I expect I will be using this tool with surprising frequency … although if I expect it, it won’t be surprising.

Be the first to comment »

Of the 4.67 gazillion pages on the Web, exactly 1.87 nano-bazillion are understandable by children. Suppose there were a convention and a service for making child-friendly versions of any site that wanted to increase its presence and value?

That was the basic idea behind our project at the MindCET Hackathon in the Desert a couple of weeks ago.

MindCET is an ed tech incubator created by the Center for Educational Technology (CET) in Israel. “Automatically generates grade-specific versions? Hahaha.”Its founder and leader is Avi Warshavsky, a brilliant technologist and a person of great warmth and character, devoted to improving education for all the world’s children. Over the ten years that I’ve been on the CET tech advisory board, Avi has become a treasured personal friend.

In Yeruham on the edge of the Negev, 14 teams of 6-8 people did the hackathon thing. Our team — to my shame, I don’t have a list of them — pretty quickly settled on thinking about what it would take to create a world-wide expectation that sites that explain things would have versions suitable for children at various grade levels.

So, here’s our plan for Onderstand.com.

Let’s say you have a site that provides information about some topic; our example was a page about how planes fly. It’s written at a normal adult level, or perhaps it assumes even more expertise about the topic. You would like the page to be accessible to kids in grade school.

No problem! Just go to Onderstand.com and enter the page’s URL. Up pops a form that lets you press a button to automatically generate versions for your choice of grade levels. Or you can create your own versions manually. The form also lets you enter useful metadata, including what school kid questions you think your site addresses, such as “How do planes fly?”, “What keeps planes up?”, and “Why don’t planes crash?” (And because everything is miscellaneous, you also enter tags, of course.)

Before I go any further, let me address your question: “It automatically generates grade-specific versions? Hahaha.” Yes, it’s true that in the 36 hours of the hackathon, we did not fully train the requisite machine learning model, in the sense that we didn’t even try. But let’s come back to that…

Ok, so imagine that you now have three grade-specific versions of your page about how planes fly. You put them on your site and give Onderstand their Web addresses as well as the metadata you’ve filled in. (Perhaps Onderstand.com would also host or archive the pages. We did not work out all these details.)

Onderstand generates a button you can place on your site that lets the visitor know that there are kid-ready versions.

The fact that there are those versions available is also recorded by Onderstand.com so that kids know that if they have a question, they can search Onderstand for appropriate versions.

Our business model is the classic “We’re doing something of value so someone will pay for it somehow.” Of course, we guarantee that we will never sell, rent, publish, share or monetize user information. But one positive thing about this approach: The service does not become valuable only once there’s lots of content. “Because sites get the kid-ready button, they get value from it”Because sites get the kid-ready button, they get value from it even if the Onderstand.com site attracts no visitors.

If the idea were to take off, then a convention that it establishes would be useful even if Onderstand were to fold up like a cheap table. The convention would be something like Wikipedia’s prepending “simple” before an article address. For example, the Wikipedia article “Airplane” is a great example of the problem: It is full of details but light on generalizations, uses hyperlinks as an excuse for lazily relying on jargon rather than readable text, and never actually explains how a plane flies. But if you prepend “simple” to that page’s URL — https://simple.wikipedia.org/wiki/Fixed-wing_aircraft — you get taken to a much shorter page with far fewer details (but also still no explanation of how planes fly).

Now, our hackathon group did not actually come up with what those prepensions should be. Maybe “grade3”, “grade9”, etc. But we wouldn’t want kids to have to guess which grade levels the site has available. So maybe just “school” or some such which would then pop up a list of the available versions. What I’m trying to say is that that’s the only detail left before we transform the Web.

The machine learning miracle

Machine learning might be able to provide a fairly straightforward, and often unsatisfactory, way of generating grade-specific versions.

“The ML could be trained on a corpus of text that has human-generated versions for kids.”The ML could be trained on a corpus of text that has human-generated versions for kids. The “simple” Wikipedia pages and their adult equivalents could be one source. Textbooks on the same subjects designed for different class levels might be another, even though — unlike the Wikipedia “simple” pages — they are not more or less translations of the same text. There are several experimental simplification applications discussed on the Web already.

Even if this worked, it’s likely to be sub-par because it would just be simplifying language, not generating explanations that creatively think in kids’ terms. For example, to explain flight to a high schooler, you would probably want to explain the Bernoulli effect and the four forces that act on a wing, but for a middle schooler you might start with the experiment in which they blow across a strip of paper, and for a grade schooler you might want to ask if they’ve ever blown on the bottom of a bubble.

So, even if the ML works, the site owner might want to do something more creative and effective. But still, simply having reduced-vocabulary versions could be helpful, and might set an expectation that a site isn’t truly accessible if it isn’t understandable.

Ok, so who’s in on the angel funding round?

Be the first to comment »

Back when I was a lad, we experienced the absurdity of life by watching as ordinary things in the world shed their meanings the way the Nazi who opens the chest in Raiders of the Lost Ark loses his skin: it just melts away.

In this experience of meaninglessness, though, what’s revealed is not some other layer beneath the surface, but the fact that all meaning is just something we make up and project over things that are indifferent to whatever we care to drape over them.

If you don’t happen to have a holy ark handy, you can experience this meaninglessness writ small by saying the word “ketchup” over and over until it becomes not a word but a sound. The magazine “Forbes” also works well for this exercise. Or, if you are a Nobel Prize winning writer and surprisingly consistently wrong philosopher like Jean Paul Sartre, perhaps a chestnut tree will reveal itself to you as utterly alien and resistant to the meaning we keep trying to throw on to it.

That was meaninglessness in the 1950s and on. Today we still manage to find our everyday world meaningless, but now we don’t see ourselves projecting meanings outwards but instead imagine ourselves to be in a computer simulation. Why? Because we pretty consistently understand ourselves in terms of our dominant tech, and these days the video cards owned by gamers are close to photo realistic, virtual reality is creating vivid spatial illusions for us, and AI is demonstrating the capacity of computers to simulate the hidden logic of real domains.

So now the source of the illusory meaning that we had taken for granted reveals itself not to be us projecting the world out from our skull holes but to be super-programmers who have created our experience of the world without bothering to create an actual world.

That’s a big difference. Projecting meaning only makes sense when there’s a world to project onto. The experience of meaninglessness as simulation takes that world away.

The meaninglessness we experience assigns the absurdity not to the arbitrariness that has led us to see the world one way instead of another, but to an Other whom we cannot see, imagine, or guess at. We envision, perhaps, children outside of our time and space playing a video game (“Sims Cosmos”), or alien computer scientists running a test to see what happens using the rules they’ve specified this time. For a moment we perhaps marvel at how life-like are the images we see as we walk down a street or along a forest path, how completely the programmers have captured the feeling of a spring rain on our head and shoulders but cleverly wasted no cycles simulating any special feeling on the soles of our feet. The whole enterprise – life, the universe, and everything – is wiped out the way a computer screen goes blank when the power is turned off.

In the spirit of the age, the sense of meaninglessness that comes from the sense we’re in a simulation is not despair, for it makes no difference. Everything is different but nothing has changed. The tree still rustles. The spring rain still smells of new earth. It is the essence of the simulation that it is full of meaning. That’s what’s being simulated. It’s all mind without any matter, unlike the old revelation that the world is all matter without meaning. The new meaninglessness is absurd absurdity, not tragic absurdity. We speculate about The Simulation without it costing a thing. The new absurdity is a toy of thought, not a problem for life.

I am not pining for my years suffering from attacks of Old School Anxiety. It was depressing and paralyzing. Our new way of finding the world meaningless is playful and does not turn every joy to ashes. It has its own dangers: it can release one from any sense of responsibility – “Dude, sorry to have killed your cat, but it was just a simulation” – and it can sap some of the sense of genuineness out of one’s emotions. But not for long because, hey, it’s a heck of a realistic simulation.

But to be clear, I reject both attempts to undermine the meaningfulness of our experience. I was drawn to philosophical phenomenology precisely because it was a way to pay attention to the world and our experience, rather than finding ways to diminish them both.

Both types of meaninglessness, however, think they are opening our eyes to the hollowness of life, when in fact they are privileging a moment of deprivation as a revelation of truth, as if the uncertainty and situatedness of meaning is a sign that it is illusory rather than it being the ground of every truth and illusion itself.

Be the first to comment »

The first in a series of six posts about my experiences learning how to train a machine learning system has just been posted here. There’s no code and no math in it. Instead it focuses on the tasks and choices involved in building one of these applications. How do you figure out what sort of data to provide? How do you get that data into the system? How can you tell when the system has been trained? What types of controls do the developers have over the outcomes? What sort of ways can I go wrong? (Given that the title of the series is “The Adventures of a TensorFlow.js n00b” the answer to that last question is: Every way.)

I was guided through this project by Yannick Assogba, a developer in the machine learning research group — People + AI Research –I’m embedded in at Google as a writer in residence. Yannick is natural born teacher, and is preternaturally patient.

The series is quite frank. I make every stupid mistake possible. And for your Schadenfreude, five more posts in this series are on their way…

.

Be the first to comment »