Joho the Bloggovernance Archives - Joho the Blog

September 19, 2017

[bkc] Hate speech on Facebook

I’m at a Very Special Harvard Berkman Klein Center for Internet & Society Tuesday luncheon featuring Monika Bickert, Facebook’s Head of Global Policy Management in conversation with Jonathan Zittrain. Monika is in charge of what types of content can be shared on FB, how advertisers and developer interact with the site, and FB’s response to terrorist content. [NOTE: I am typing quickly, getting things wrong, missing nuance, filtering through my own interests and biases, omitting what I can’t hear or parse, and not using a spelpchecker. TL;DR: Please do not assume that this is a reliable account.]

Monika: We have more than 2B users…

JZ: Including bots?

MB: Nope, verified. Billions of messages are posted every day.

[JZ posts some bullet points about MB’s career, which is awesome.]

JZ: Audience, would you want to see photos of abused dogs taken down. Assume they’re put up without context. [It sounds to me like more do not want it taken down.]

MB: The Guardian covered this. [Maybe here?] The useful part was it highlighted how much goes into the process of deciding these things. E.g., what counts as mutilation of an animal? The Guardian published what it said were FB’s standards, not all of which were.

MB: For user generated content there’s a set of standards that’s made public. When a comment is reported to FB, it goes to a FB content reviewer.

JZ: What does it take to be one of those? What does it pay?

MB: It’s not an existing field. Some have content-area expertise, e.g., terrorism. It’s not a minimum wage sort of job. It’s a difficult, serious job. People go through extensive training, and continuing training. Each reviewer is audited. They take quizzes from time to time. Our policies change constantly. We have something like a mini legislative session every two weeks to discuss proposed policy changes, considering internal suggestions, including international input, and external expert input as well, e.g., ACLU.

MB: About animal abuse: we consider context. Is it a protest against animal cruelty? After a natural disaster, you’ll see awful images. It gets very complicated. E.g., someone posts a photo of a bleeding body in Syria with no caption, or just “Wow.” What do we do?

JZ: This is worlds away from what lawyers learn about the First Amendment.

MB: Yes, we’re a private company so the Amendment doesn’t apply. Behind our rules is the idea that “You don’t have to agree with the content, but you should feel safe”FB should be a place where people feel safe connecting and expressing themselves. You don’t have to agree with the content, but you should feel safe.

JZ: Hate speech was defined as an attack against a protected category…

MB: We don’t allow hate speech, but no two people define it the same way. For us, it’s hate speech if you are attacking a person or a group of people based upon a protected characteristic — race, gender, gender identification, etc. —. Sounds easy in concept, but applying it is hard. Our rule is if I say something about a protected category and it’s an attack, we’d consider it hate speech and remove it.

JZ: The Guardian said that in training there’s a quiz. Q: Who do we protect: Women drivers, black children, or white men? A: White men.

MB: Not our policy any more. Our policy was that if there’s another characteristic beside the protected category, it’s not hate speech. So, attacking black children was ok but not white men, because of the inclusion of “children.” But we’ve changed that. Now we would consider attacks on women drivers and black children as hate speech. But when you introduce other characteristics such as profession, it’s harder. We’re evaluating and testing policies now. We try marking content and doing a blind test to see how it affects outcomes. [I don’t understand that. Sorry.]

JZ: Should the internal policy be made public?

MB: I’d be in favor of it. Making the training decks transparent would also be useful. It’s easier if you make clear where the line is.

JZ: Do protected categories shift?

MB: Yes, generally. I’ve been at FB for 5.5 yrs, in this are for 4 yrs. Overall, we’ve gotten more restrictive. Sometimes something becomes a topic of news and we want to make sure people can discuss it.

JZ: Didi Delgado’s post “all white people are racist” was deleted. But it would have been deleted if had said that all black people are racist, right?

MB: Yes. “If it’s a protected characteristic, we’ll protect it”If it’s a protected characteristic, we’ll protect it. [Ah, if only life were that symmetrical.]

JZL How about calls to violence, e.g., “Someone shoot Trump/Hillary”? If you think it should be taken down. [Sounds like most would let it stand.]

JZ: How about “Kick a person with red hair.” [most let it stand]

JZ: “How about: To snap a bitch’s neck, make sure to apply all your pressure to the middle of her throat.” [most let it stand][fuck, that’s hard to see up on the screen.]

JZ: “Let’s beat up the fat kids.” [most let it stand]

JZ: “#stab and become the fear of the Zionist” [most take it down]

MB: We don’t allow credible calls for violence.

JZ: Suppose I, a non-public figure, posted “Post one more insult and I’ll kill you.”

MB: We’d take that down. We also look at the degree of violence. Beating up and kicking might not rise to the standard. Snapping someone’s neck would be taken down, although if it were purely instructions on how to do something, we’d leave it up. “Zionist” is often associated with hate speech, and stabbing is serious, so we’d take them down. We leave room for aspirational statements wishing some bad thing would happen. “Someone should shoot them all” we’d count as a call to violence. We also look for specifity, as in “Let’s kill JZ. He leaves work at 3.” We also look at the vulnerability of people; if it’s a dangerous situation,
we’ll tend to treat all such things as calls to violence, [These are tough questions, but I’m not aligned with FB’s decisions on this.]

JZ: How long does someone spend reviewing this stuff?

MB: Some is easy. Nudity is nudity, although we let breast cancer photos through. But a beheading video is prohibited no matter what the context. Profiles can be very hard to evaluate. E.g., is this person a terrorist?

JZ: Given the importance of FB, does it seem right that these decisions reside with FB as a commercial entity. Or is there some other source that would actually be a relief?

MB: “We’re not making these decisions in a silo”We’re not making these decisions in a silo. We reach out for opinions outside of the company. We have Safety Advisory Board, a Global Safety Network [got that wrong, I think], etc.

JZ: These decisions are global? If I insult the Thai King…

MB: That doesn’t violate our global community standard. We have a group of academics around the world, and people on our team, who are counter-terrorism experts. It’s very much a conversation with the community.

JZ: FB requires real names, which can be a form of self-doxxing. Is the Real Name policy going to evolve?

MB: It’s evolved a little about what counts as their real name, i.e., the name people call you as opposed to what’s on your drivers license. Using your real name has always been a cornerstone of FB. A quinessential element of FB.

JZ: You don’t force disambiguation among all the Robert Smiths…

MB: When you communicate with people you know, you know you know them. “We don’t want people to be communicating with people who are not who you think they are”We don’t want people to be communicating with people who are not who you think they are. When you share something on FB, it’s not public or private. You can choose which groups you want to share it with, so you know who will see it. That’s part of the real name policy as well.

MB: We have our community standards. Sometimes we get requests from countries to remove violations of their law, e.g., insults to the King of Thailand. If we get such a request, if it doesn’t violate the standards, we look if the request is actually about real law in that country. Then we ask if it is political speech; if it is, to the extent possible, we’ll push back on those requests. E.g., Germans have a little more subjectivity in their hate speech laws. They may notify us about something that violates those laws, and if it does not violate our global standards, we’ll remove it in Germany only. (It’s done by IP addresses, the language you’re using, etc.) When we do that, we include it in our 6 month reports. If it’s removed, you see a notice that the content is restricted in your jurisdiction.

Q&A

Q: Have you spoken to users about people from different cultures and backgrounds reviewing their content?

A: It’s a legitimate question. E.g., when it comes to nudity, even a room of people as homogenous as this one will disagree. So, “our rules are written to be very objective”our rules are written to be very objective. And we’re increasingly using tech to make these decisions. E.g., it’s easy to automate the finding of links to porn or spam, and much harder for evaluating speech.

Q: What drives change in these policies and algorithms?

A: It’s constantly happening. And public conversation is helpful. And our reviewers raise issues.

Q: a) When there are very contentious political issues, how do you prevent bias? b) Are there checks on FB promoting some agenda?

A: a) We don’t have a rule saying that people from one or another country can review contentious posts. But we review the reviewers’ decisions every week. b) The transparency report we put out every six months is one such check. If we don’t listen to feedback, we tend to see news stories calling us out on it.

[Monika now quickly addresses some of the questions from the open question tool.]

Q: Would you send reports to Lumen? MB: We don’t currently record why decisions were made.

Q: How to prevent removal policies from being weaponized but trolls or censorious regimes? MB: We treat all reports the same — there’s an argument that we shouldn’t — but we don’t continuously re-review posts.

JZ: For all of the major platforms struggling with these issues, is it your instinct that it’s just a matter of incrementally getting this right, bringing in more people, continue to use AI, etc. OR do you think sometimes that this is just nuts; there’s got to be a better way.

There’s a tension between letting anyone see what they want, or have global standards. People say US hates hate speech and the Germans not so much, but there’s actually a spectrum in each. The catch is that there’s content that you’re going to be ok seeing but we think is not ok to be shared.

[Monika was refreshingly direct, and these are, I believe, literally impossible problems. But I came away thinking that FB’s position has a lot to do with covering their butt at the expense of protecting the vulnerable. E.g., they treat all protected classes equally, even though some of us — er, me — are in top o’ the heap, privileged classes. The result is that FB applies a rule equally to all, which can bring inequitable results. That’s easier and safer, but it’s not like I have a solution to these intractable problems.]

1 Comment »

November 22, 2016

[liveblog][bkc] Scott Bradner: IANA: Important, but not for what they do"

I’m at a Berkman Klein [twitter: BKCHarvard] talk by Scott Bradner about IANA, the Internet Assigned Names Authority. Scott is one of the people responsible for giving us the Internet. So, thanks for that, Scott!

NOTE: Live-blogging. Getting things wrong. Missing points. Omitting key information. Introducing artificial choppiness. Over-emphasizing small matters. Paraphrasing badly. Not running a spellpchecker. Mangling other people’s ideas and words. You are warned, people.

Scott begins by pointing to the “absurdity” of Ted Cruz’s campaign
to prevent the “Internet giveaway.”“ The idea that “Obama gave away the Internet” is “hooey,”” The idea that “Obama gave away the Internet” is “hooey,” says Scott.

IANA started with a need to coordinate information, not to control it, he says. It began with the Network Working Group in 1968. Then Requests for Comments (RFC) in 1969. . The name “IANA” showed up in 1988, although the function had begun in 1972 with coordinating socket numbers. The Domain Name System made IP addresses easier to use, including the hierarchical clustering under .com, .org, etc.

Back to the beginning, computers were too expensive for every gov’t department to have one. So, ARPA wanted to share large and expensive computers among users. It created a packet-based network, which broke info up into packets that were then transmitted. Packet networking was the idea of Paul Baran at RAND who wanted a system that would survive a nuclear strike, but the aim of that network was to share computers. The packets had enough info to make it to their destinations, but the packet design made “no assumptions about the underlying transport network.” No service guarantees about packets making it through were offered. The Internet is the interconnection of the different networks, including the commercial networks that began showing up in the 1990s.

No one cared about the Net for decades. To the traditional telecom and corporate networking people, it was just a toy—”No quality of service, no guarantees, no security, no one in charge.” IBM thought you couldn’t build a network out of this because their definition of a network — the minimal requirements — was different. “That was great because it meant the regulators ignored us.”

The IANA function went into steady state 1984-1995. It did some allocating of addresses. (When Scott asked Jon Postel for addresses for Harvard, Postel sent him some; Postel was the one-person domain allocation shop.) IANA ran it for the top level domains.

“The Internet has few needs,” Scott says. It’s almost all done through collaboration and agreement. There are no requirements except at a very simple level. The only centralized functions: 1. We have to agree on what the protocol parameters are. Machines have to understand how to read the packet headers. 2. We have to allocate blocks of IP addresses and ASN‘s. 3. We have to have a single DNS, at least for now. IANA handles those three. “Everything else is distributed.” Everything else is collaboration.

In 1993, Network Solutions was given permission to start selling domain names. A domain cost $100 for 2 yrs. There were were about 100M names at that point, which added up to real money. Some countries even started selling off their TLD’s (top level domains), e.g., .tv

IANA dealt with three topics, but DNS was the only one of interest to most people. There was pressure to create new TLDs, which Scott thinks doesn’t solve any real problems. That power was given to ISOC, which set up the International Ad-Hoc Committee in 1996. It set up 7 new TLDs, one of which (.web) caused Image Online Design to sue Postel because they said Postel had promised it to them. The Dept. of Commerce saw that it needed to do something. So they put out an RFC and got 400+ comments. Meanwhile, Postel worked on a plan for institutionalizing the IANA function, which culminated in a conference in Jan 1998. Postel couldn’t go, so Scott presented in his stead.

Shortly after that the Dept of Commerce proposed having a private non-profit coordinate and manage the allocation of the blocks to the registries, manage the file that determines TLDs, and decide which TLDs should exist…the functions of IANA. “There’s no Internet governance here, simply what IANA did.”

There were meetings around the world to discuss this, including one sponsored by the Berkman Center. Many of the people attending were there to discuss Internet governance, which was not the point of the meetings. One person said, “Why are we wasting time talking about TLDs when the Internet is going to destroy countries?” “Most of us thought that was a well-needed vacuum,” says Scott. We didn’t need Internet governance. We were better off without it.

Jon Postel submitted a proposal for an Internet Corporation for Assigned Names and Numbers (ICANN). He died of a heart attack shortly thereafter. The Dept. of Commerce accepted the proposal. In Oct 1998 ICANN had its first board meeting. It was a closed meeting “which anticipated much of what’s wrong with ICANN.”

The Dept of Commerce had oversight over ICANN but its only power was to say yes or no to the file that lists the TLDs and the IP addresses of the nameservers for each of the TLDs.” “That’s the entirety of the control the US govt had over ICANN. “In theory, the Dept of Commerce could have said ‘Take Cuba out of that file,’ but that’s the most ridiculous thing they could have done and most of the world could have ignored them.” The Dept of Commerce never said no to ICANN.

ICANN institutionalizes the IANA. But it also has to deal with trademark issues coming out of domain name registrations, and consults on DNS security issues. “ICANN was formed as a little organization to replace Jon Postel.”

It didn’t stay little. ICANN’s budget went from a few million bucks to over $100M.“ “That’s a lot of money to replace a few competent geeks.”” “That’s a lot of money to replace a few competent geeks.” It’s also approved hundreds of TLDs. The bylaws went from 7,000 words to 37,000 words. “If you need 37,000 words to say what you’re doing, there’s something wrong.”

The world started to change. Many govts see the Net as an intrinsic threat.

  • In Sept. 2001, India, Brazil, and South Africa proposed that the UN undertake governance of the Internet.

  • Oct 2013: After Snowden, the Montevideo Statement on the Future of Internet Cooperation proposing moving away from US govt’s oversight of IANA.

  • Apr. 2014: NetMundial Initiative. “Self-appointed 25-member council to perform internet governance.”

  • Mar. 2014: NTIA announces its intent to transition key domain name functions.

The NTIA proposal was supposed to involve all the stakeholders. But it also said that ICANN should continue to maintain the openness of the Internet…a function that ICANN never had. Openness arises from the technical nature of the Net. NTIA said it wouldn’t accept an inter-governmental solution (like the ITU) because it has to involve all the stakeholders.

So who holds ICANN accountable? They created a community process that is “incredibly strong.” It can change the bylaws, and remove ICAN directors or the entire board.

Meanwhile, the US Congress got bent out of shape because the US is “giving away the Internet.” It blocked the NTIA from acting until Sept. 2016. On Oct. 1 IANA became independent and is under the control of the community. “This cannot be undone.” “If the transition had not happened, forces in the UN would likely have taken over” governance of the Internet. This would have been much more likely if the NTIA had not let it go. “The IANA performs coordination functions, not governance. There is no Internet governance.”

How can there be no governance? “Because nobody cared for long enough that it got away from them,” Scott says. “But is this a problem we have to fix?”

He leaves the answer hanging. [SPOILER: The answer is NO]

Q&A

Q: Under whom do the IRI‘s [Internationalized Resource Identifier] operate?

A: Some Europeans offered to take over European domain names from Jon Postel. It’s an open question whether they have authority to do what they’re doing Every one has its own policy development process.

Q: Where’s research being done to make a more distributed Internet?

A: There have been many proposals ever since ICANN was formed to have some sort of distributed maintenance of the TLDs. But it always comes down to you seeing the same .com site as I do — the same address pointing to the same site for all Internet users. You still have to centralize or at least distribute the mapping. Some people are looking at geographic addressing, although it doesn’t scale.

Q: Do you think Trump could make the US more like China in terms of the Internet?

A: Trump signed on to Cruz’s position on IANA. The security issue is a big one, very real. The gut reaction to recent DDOS
attacks is to fix that rather than to look at the root cause, which was crappy devices. The Chinese government controls the Net in China by making everyone go through a central, national connection. Most countries don’t do that. OTOH, England is imposing very strict content

rules that all ISPs have to obey. We may be moving to a telephony model, which is a Westphalian
idea of national Internets.

Q: The Net seems to need other things internationally controlled, e.g. buffer bloat. Peer pressure seems to be the only way: you throw people off who disagree.

A: IANA doesn’t have agreements with service providers. Buffer bloat is a real issue but it only affects the people who have it, unlike the IoT DDOS attack that affected us all. Are you going to kick off people who’s home security cameras are insecure?

Q: Russia seems to be taking the opposite approach. It has lots of connections coming into it, perhaps for fear that someone would cut them off. Terrorist groups are cutting cables, botnets, etc.

A: Great question. It’s not clear there’s an answer.

Q: With IPv6 there are many more address spaces to give out. How does that change things?

A: The DNS is an amazing success story. It scales extremely well … although there are scaling issues with the backbone routing systems, which are big and expensive. “That’s one of the issues we wanted to address when we did IPv6.”

Q: You said that ICANN has a spotty history of transparency. What role do you think ICANN is going to play going forward? Can it improve on its track record?

A: I’m not sure that it’s relevant. IANA’s functions are not a governance function. The only thing like a governance issue are the TLDs and ICANN has already blown that.

Comments Off on [liveblog][bkc] Scott Bradner: IANA: Important, but not for what they do"

July 10, 2011

Elliot Noss — an Internet good guy, and my friend — finds additional significance in ICANN’s June decision to open up new dot-whatever names to anyone willing to pay a very hefty price:

1 Comment »

November 16, 2009

UN’s Internet Governance Forum censors a mild mention of censorship

Holy cow!

The Open Net Initiative, a group that monitors government filtering (= censorship) of the Internet held a book launch at the United Nations-sponsored Internet Governance Forum in Sharm El Sheik. A poster for the book — Access Controlled — contained the sentence: “The first generation of Internet controls consisted largely of building firewalls at key Internet gateways; China’s famous ‘Great Firewall of China’ is one of the first national Internet filtering systems.”

This statement was so objectionable, so outrageous, such a violation of common decency, such a hateful expression, such an offense to the tender sensibilities of UN diplomats that it must not ever be uttered. Security guards were sent to take the poster down.

If the people who want to govern the Internet think that’s beyond the pale of free speech, what the hell are they going to do with the rest of the Internet?

And, by the way, if you want to see what it looks like when UN diplomats take bold action, watch this video of the take-down itself.

[Source, video statement by ONI, BoingBoingage]

(Disclosure: the Berkman Center is a member of ONI.)

2 Comments »

April 18, 2009

Jonathan Zittrain on Facebook’s open-ish governance

JZ has a terrific post on the new participatory governance announced by Facebook. I found myself nodding as I read it, and sometimes even rubbing my chin thoughtfully. It is a fascinating experiment.

[Tags: ]

Comments Off on Jonathan Zittrain on Facebook’s open-ish governance

January 17, 2009

Leadership and the Interregnum

I hope someday an historian writes a book called The Interregnum that looks at the period between the election and inauguration of Barack Obama. Not since the Cuban Missile Crisis had us huddled waiting for events to resolve have I had such a palpable sense of history. But now, instead of parsing every car horn as the start of a nuclear siren, I am ready for hope.

The stew of emotions is rich.

Hope itself is encompassing. It isn’t even an emotion. It’s a full-body experience, including cognition, anticipation, dedication, and spirit. In this case, hope is social. It’s not me trusting looking into the eyes of my Maker. It’s us relying on us.

Then there’s patriotism. I’ve always been more interested in the reasons that justify patriotism than in patriotism itself. But now I’m proud of how we are responding to this person we improbably elected.

There’s fear. I want my children to have the same opportunities I’ve been privileged to have. That is far from guaranteed. It isn’t even likely.

But The Interregnum will make for compelling reading most of all because it is the story of two people who could not be more different as people and as leaders.

Although I’ve been furious at President Bush for years, I had no idea I’ve actually been holding some back. I didn’t think I had any more to give. But then George Bush began his round of farewells.

Whatever someone says s/he is is exactly what that person is not. If your boss says, “I’m all about honesty,” then your boss is a liar. “For me, accountability is the main thing” means your boss is a swindler.

Bush told us he is all about compassion.

As Bush has put forward his self-explanation and justification in this past week, it’s become clear how incapable he is of seeing things from someone else’s point of view. With millions of refugees created in Iraq, he says his mistake was in posing in front of that “Mission Accomplished” sign. In the face of Katrina’s refugees, Bush thinks his mistake was not arriving on scene for his photo opp earlier. As Jon Stewart said, “You have no idea why people are angry at you, do you?”

I don’t think this is due to narcissism on Bush’s part. I think it’s part and parcel of his lack of intellectual curiosity. He’s a tiny man on a vast stage who simply can’t think past himself and what he sees at the moment. It doesn’t matter how large the stage becomes, his tiny circle of light never expands.

Bush provides us with the final and perfect exemplar of how our American idea of leadership, in politics and business, has gone wrong. We’ve taken leadership as a personality trait. Bush thinks he’s a leader because he made unpopular decisions and stuck by them. Leadership to him is a matter of character. If that’s all leadership is, then we’re better off without leaders — people empty of anything except a random resolve to do something and then keep doing it.

What’s missing is the idea that leaders need to be responsive to the reality of the world, the reality of the conflicting needs of the led, and the reality of suffering. Leaders may sometimes need to draw a clear line, but they must always recognize that the simplicity some decisions require masks an awful complexity.

In the interregnum, Bush has revealed himself as a buffoon blind to the tragedy he has hosted, while Obama has been showing us what leadership is about by bringing us to what is best in ourselves — as individuals, and, most of all, together.

I am ready for release from the shame and anger of the Bush years. I am so ready for the interregnum to end. [Tags: ]

6 Comments »

June 19, 2008

Governance discussion at Supernova

Berkman sponsored a discussion at Supernova on governance, the topic of the Publius project. Here are Ross Mayfield’s notes. Sounds like it was a terrific session, with great people on the panel and joining in from the audience. [Tags: ]

Comments Off on Governance discussion at Supernova

June 6, 2008

Open education and Publius

Berkman‘s Publius project keeps rolling along. There’s already lots of excellent stuff there, exploring how the Net is constituting its own governance mechanisms and norms. For example, today Peter Suber and Melissa Hagemann discuss open access, science, research, and education. But you can just browse through the topics and be pretty sure you’ll hit on something well worth reading.

[Tags: ]

1 Comment »

May 16, 2008

Me on implicit governance and the Publius Project

Supernova has posted a 15 minute vcast interview with me, by Howard Greenstein, about the Berkman Publius Project and my op-ed in it about why tacit governance is usually better than getting all explicit about stuff.

[Tags: ]

1 Comment »

May 13, 2008

Publius is publicus!

The Berkman Center’s Publius Project is now live. There you’ll find essays on the Internet’s “constitutional moments,” even though most of those moments do not involve a written constitution … which makes the topic all the more interesting. (My contribution, on tacit governance, is here.)

[Tags: ]

Comments Off on Publius is publicus!