New Site Index

The free site index I've been happily using for years, Atomz, tells me I've gone over the 500 file limit. (I seem to have something like 1,600 pages on my site. Seems wrong to me, but, well, whatever.) Being a cheap bastard — and the fact that you have to call Atomz for a quote is a little off-putting — I'm looking for a new, free index. Care to kick the tires of FreeFind?

Blog-free Wednesday

I'll be on the road all of Thursday. I may not have time to check my email much less blog. Oh the horror!

Ernie on the Singularity

Ernie the Attorney has a rollicking piece on why The Singularity — the moment when the humanity-robotronics combo at last exceeds the merely human — is to be feared. For example:

Unless technology is somehow going to make us better thinkers then we are going to carry into this brave (and sudden) new world our very strong tendency to be led by fear and dogmatic beliefs (a/k/a "bias and prejudice"). The only way we won't pollute this new "Singular World" with our pathetic cognitive weaknesses will be if only the "intellectually pure" are admitted. Or let's put it this way, you can't send a "representative cross-section" of our current world there, that's for sure.

(This makes the second time today that I've heard Kurzweilians accused of replicating religion's mistakes.)

Steve Talbott on Taking the You out of Eugenics

Steve Talbott picks an excellent argument with Bill McKibben who argues in his new book, "Enough: Staying Human in an Engineered Age", that once we're able to alter our children's DNA, our offspring will have no sense of a self that is their own. Talbott's point is that McKibben gives too much credit to the power of DNA to determine who we are:

By appearing to validate the scientist's (and the public's) conviction that we are our protein-producing DNA, McKibben is assisting the engineers' program. For while his commendable aim is to convince us to pull back from the eugenic brink, the fact is that those who think they are their DNA are exactly the ones who will clamor for a new and improved self, or at least for new and improved children.

Will the genetic engineers make our lives meaningless? This is ever so close to the truth, yet light years away from it. No one can, in absolute terms, rob someone else of meaning. What makes life meaningless is our rejection of meaning — a rejection we have already given expression to when we conceive ourselves as the product of DNA "mechanisms".

Talbott then goes on to the larger point:

That the worshippers of machinery, efficiency, and power are engaged today in a fateful assault upon the human being is beyond all doubt. McKibben performs a valuable service by documenting this assault for a large audience from the mouths of the commandos carrying it out. There is no shortage of testimony.

And then the truth:

If it's true, as I have suggested, that we unavoidably affect each other's destinies — for ill, but also for good — then everything hinges upon our understanding of this mutuality. And the first thing to grasp is that healthy human exchange is, and is essentially, a matter of mutuality. We are called to engage each other in a mutually respectful dance or conversation, which is very different from unilateral manipulation. Conversation or manipulation: *this* is the decisive distinction.

Finally, the Parthian shot at the transhumanist extropians, et al.:

You can't read the futuristic scenarios and personal hopes of the re-engineers of humanity without being struck by the utter childishness of it all. Genetic modifications that will save us from the necessity of bodily excretion; nano-contrived plants that look exactly like orchids but can grow in frigid climes; robots that wait on us like slaves; a cyber-nano- genetically engineered "elite race of people who are smart, agile, and disease-resistant"; nanobot swarms able to wander the human bloodstream and keep us eternally healthy; technological horns of plenty that will convert every "desolate" village into "a Garden of Eden, with widescreen TVs and cappuccino machines for all"....and so on ad infinitum.

And many of these visions come from the same people who delight in ridiculing the "childish hopes" of the traditionally religious!

I'm a-liking Talbott...

EULA Poll

CodeMaster.com offers the results of this week's poll.

Well, that can't be scientific. I don't believe for a second that out of 1,339 respondents, there are 18 people who actually read and stick to those stupid EULAs, much less that 0.4% bother to always read them but then don't stick to them anyway. What's the point of that??

Sen. Santorum's Guide to Sex

Hilarious call from the Scarlet Pimpernel to Senator "Sanctorum" Santorum.

[ETech] Wrap-up and Overview

This was a terrific conference for the two most important reasons: Terrific attendees and more than enough sessions to learn from.

If a theme emerged, I think it was emergence itself. Most of the presentations that most excited me played off of this in one way or another: Eric Bonabeau on emergence in nature, Clay Shirky on why groups resist explicit constitutions, the social software track, Alan Kay's "broadband collaboration" environment which is interesting because of the ease with which participants can set new creatures loose into it. Emergence is the way in which bottom-up organization happens.

There were, of course, other highlights. I loved Brewster Kahle's Internet Bookmobile project, Eric Blossom nd Matt Ettus' GNU radio, Ben Hammersley's proselytizing for threadsML/ENT, and Marc Smith's Netscan stuff. And more. And what could be better than getting to hang for three days with some of the smartest technoids around?

Things not to like? Yeah, sure: Not enough scheduled schmoozing time. Too many excellent sessions in competing time slots. Some dud keynotes from vendors. Terrible lunches. And way too many white men.

That aside, it was a first-rate conference.

Why social software now?

A small brouhaha is brewhaha-ing over whether "social software" is mere hype. (See Frank Paynter, for example.) After all, the category is about as broad as "software for people" and includes technology as old as holding hands.

And yet it's the thing I came away from the O'Reilly Conference most excited about.

First, I consider social software actually to be emergent social software. That narrows the field to software that enables groups to form and organize themselves. Yes, it's still broad but at least it's not coextensive with any software that has a user interface.

Second, it doesn't much matter to me whether the software is new or old. I'm excited about the fact that that type of software is now being recognized (i.e., "hyped") as important. And my question is: Given that most of the software is old, why is this category now becoming hot?

Sure, in part it's because consultants (like, um, me) and writers (like, um um, me) now have something new to flap their gums about. But, more important, I think and hope it's because the central idea behind emergent social software is now becoming acceptable: We're beginning to think that letting groups start without rules, letting people organize themselves as they see fit at the moment and in that context, is actually a good idea and not just a waste of time, a hippy dream, or a threat. Gosh, maybe a wiki isn't only an invitation to vandals but is a useful way for people to collaborate! But to think so means trusting groups of people to work well together even when their choke collars are undone.

Much emergent social software may be old hat, but that now we're willing to recognize its value is pretty damn exciting.

Orlowski Again

Jonathan at Way.Nu rants right back at Andrew Orlowski's latest villification of blogging. Orlowski seems to be descending through the rings of his own private Inferno column by column.

No, I'm not putting in a link to Orlowski's piece. You can get there through Jonathan. I don't want to reward Orlowski by pushing him up the blog rank. (This is why we need Kevin Marks' "vote" attribute that lets us specify that although we're linking to something, we don't like it.)

My ambiguous breakfast with Esther

I had breakfast with Esther Dyson this morning (yes, lucky me) which she blogged. We talked about the importance of ambiguity, and Esther pushed me off my stuck point. I'm not sure exactly where that leaves me, but that's the best place to be. So, thanks, Esther.

[Etech] Google Keynote

Craig Silverstein is talking about how Google Inc. works.

Some people think a company has to be evil to some degree to be successful, to play hardball. Craig's glad that Google doesn't think that way.

He's going through general principles. E.g., "Good ideas come from everywhere." (His example is the TouchGraph Google visual browser.) "Communications is key," etc. It's great to hear a company that's walking the walk, but I'd rather have more of a drill down into the development process.

Ah, now he's talking about the documents Google uses to organize engineering, e.g., their weekly engineering report. And they use blogs. (His example is evhead, Evan Williams' blog.) He says no one speculated when they bought Blogger.com that they might use blogging internally. And now it's working out great.

Google maintains a wiki-like page where people can contribute ideas for enhancements. They hold informal brainstorming sessions to evaluate the ideas.

They used to launch betas on the site and wait for feedback. Now they put it through some initial testing. They put experiments up at labs.google.com. (Craig particularly likes Google Sets. "It's particularly useful if you can't remember all the names of the seven dwarves.)

Craig says that Google is hiring. You can practically hear 500 people mentally composing their resumes.

(His slides are as simple as Google's UI.)

I asked why Google only indexes 3 billion pages. Craig says that that's how many they can crawl in a month.

Esther Dyson asks what they'll be doing with their purchase of Applied Semantics, but Craig is amusingly evasive.

Question: The RIAA just sued a college because a local search engine discovered students were sharing MP3s. But Google is even better at finding MP3s. How worried are you?

Answer: We don't let people search for music precisely for these legal reasons. We don't want to make it easy to find illegal information. It's hard to reconcile respect for intellectual property and make all the world's information available.

Question: What are the barriers to all companies being like Google?

Answer: In part it's because the enabling technology is new. Also, it takes a lot of work.

[ETech] Friday: Microsoft

L. F. (Felipe) Cabrera, Ph.D., is talking about Microsoft Web services. He's trying too hard to convince the crowd that he's a fellow geek even as he re-cycles a generic Microsoft slide set about Web services. (We know it's recycled since he keeps telling us that we're too smart to need this slide or that. )

Four design principles: Modular and composable; general purpose, standards-based and federated. Felipe elaborates on the importance of federation: "no central point of administration, control or failure." "Federation forces you to respect all these different degrees of autonomy." He's stressing this obviously because Microsoft has been late to the federation party. Microsoft is focused on standards and interoperability, he says.

He's begging us to interrupt him with questions. He thinks we're "not awake." [Or, perhaps he's just not being interesting enough.]

Now Tim O'Reilly is saying that the barrier to entry is too high because there is such a high stack of specifications. Felipe says that IP was once a technical hurdle. There's a stack of specs because people want to do different things with distributed computing. Microsoft wants us to get to the point where the stack of specs is just taken for granted.

Judi Clark asks if Microsoft is now willing to work on open standards. "Absolutely," says Felipe. "We hire more people to work on standards than lawyers," he says to laughter and applause. But then he appends: "That's Felipe's perception." And, he adds, "A clean design is better than a committee design." [Ominous!]

Greg Elin asks if GPL is a guarantee of acces to code. What is the documented promise that what Microsoft says today will be valid in 2-3 years.

Felipe responds: "What's GPL?" and totally loses the audience.

Question: Microsoft isn't there with interoperability.

Answer: Yeah, it drives us crazy.

Question: In defense of MSFT, much of the problem comes from the XML Schema spec which is a nightmare.

Saman Far: How about having a reference implementation available in an open source way? Have you considered having the source available for the protocol stack implementations?

Answer: It has been hotly debated. I'm just a techie.

[ETech] Meg Hourihan & Microsoft Data Mining

Tough choice among sessions! A Microsoft researcher is talking about social software, Mitch Kapor is talking about Chandler, and Meg "Megnut" Hourihan is talking about "From the Margins of the Writable Web." Meg's always interesting and I love her title, so I'm here.

The tools for reading weblogs aren't as developed as for writing them. Meg points to sites doing interesting things. E.g., weblogs that are tied to geographic areas. (You can put your geographic information into your blog via geourl.org.)

Also, sites are getting more explicit in their social relationships. E.g., create an OPML file of all your friends and put it in your weblog...

This is great stuff, but I can't read the slides from in back and thus can't get the URLs. I'll get them from Meg's site when she posts them. You should too. On to the Microsoft guy...

Marc Smith is talking about Netscan, a project for data mining newsgroups to see what we can learn about their social organization. For example, the number of cross-posted threads can indicate whether the newsgroup needs to fork. And 67% of Usenet threads have only two messages. Does this indicate success or failure? E.g., a customer support group wants short threads. How do you tell? One guy posted 95 times and every one was a reply. And posted 25 out of 26 days. He's likely to be a high value "answer person."

He thinks this type of analysis will be used by professional organizations trying to keep their discussion lists healthy. What makes an online community healthy? He says that you should look at things like time to reply, number of posts, percentage of messages replied to, retention of leaders, etc. He shows a user-friendly web page that Microsoft Research is trying to get Microsoft to build.

He demonstrates reading bar codes to get discussion threads about the bar-coded object. (It's called AURA: Advanced User Research Application)

Fascinating. And Smith is a terrific presenter, getting laughter and applause along the way...tough for a Microsoft guy in this crowd.

[ETech] Ben Hammersley: Mail Threads

The Semantic Web, Ben says entertainingly, isn't going to happen because the Web is too messy. So what does already have the data that's required? A mailing list. Tons of metadata already there.

With threadsML (yeah!) you can capture threads. It's "RSS 1.0 with extra toys." It includes some Dublin Core metadata (metadata standard for documents). "Mod threading" lets you point to things that are children of the object.

This addresses the obsolescence of conversations.

The killer problem is: Subjects (topics). Why? Because subject lines are often not related to the topic. A solution is to adopt one of the topic hierarchies, e.g., Yahoo or Dmoz. But they're culturally relative and brittle. An alternative is to create your own hierarchical ontology. E.g., Easy News Topics for RSS 2.0 (RSS 2.0 is RSS without the RDF components.) ENT was created by Matt Mower and Paolo Valdemarin. ENT points to a topic map and lets you specify a topic within the message. Matt and Paolo have today released an aggregator that reads the ENT topics. But Ben's not sure how it's going to work because even simple threads get long lists of topics.

What do you do about the proliferation of classification systems? When someone links A to B, the software can look up the topics that B declares, and it can assume that that's a vote for saying that the topic of A is similar to the topic of B. [Interesting. This would work across languages.]

Marc Canter, who helped revive interest in threadsML, asks what are the benefits of having this sort of thing. Audience suggestions:

- Move threads from email to discussion boards to wiki, etc.

- Do it on one computer, moving between email systems, etc.

- Query it

- Handle not just text

- Link threads so the "backstory" is available

- Use email interface to read blogs, wikis, mailing list archives

- Connect reviews

(Disclosure: I've been involved with threadsML since the beginning, so I'm pretty stoked about this session. In fact, do a whois on threadsml.com... :)

[ETech] Clay Shirky Keynote

Clay is going to talk about a pattern in social software that consistently emerges: Groups are their own worst enemy.

Prior to the Internet we had lots of ways of doing point-to-point and one-to-many communication. Before the Internet, we didn't have a good way to do "ridiculously easy group forming." Now we do. But the patterns that emerge are due not to the technology but to how humans behave in groups.

Part 1: Why groups are their own worst enemy.

Wilfred Bion wrote in the middle of the 20th century. He believed that we are irreducibly social and individual. Bion found three patterns. First, groups engage in flirtatious sex talk among pairs, no matter what the ostensible group purpose. Second, they identify and villify enemies. Third, they nominate and venerate a hero beyond critique. All three of these are obvious on the Internet also. Groups "sandbag their sophistcated goals with these basic urges."

Bion concluded that group structure is necessary to defend the group from itself. It exists to keep a group on track. Group structure defends a group from the actions of its own members. For example: CommuniTree, a BBS in the '70s that failed when adolescents posted obscenities all over.

Groups come to a constitutional crisis where they not only need rules, they need rules about making rules.

Part 2: Why now?

If these things have been happening forever, why is it important now?

Observationally, there's a revolution in social software happening. (Social software: software that supports groups.) Small groups are different than big groups. Now we're getting weblogs and wikis and platform stuff that lets us try new things rapidly that support small to midsize groups, e.g., RSS.

Why now? Really: Why did it take so long? Why did it take so long to get weblogs, for example? WEW could have weblogs when we had the first forms-capable browser. Answer: It just took us that long to figure it out.

Second, this stuff is truly Web-native, unlike, say, Notes with a lightweight Web interface.

Third, we can easily put stuff together. E.g., Joi Ito's conference call with a chat attached and then a wiki. It was a broadband conference call, but it's a simple little thing.

Fourth, ubiquity. We can now assume, in many situations, all people have access to the Web. "All is a different kind of amount than most." And within a meeting, we can begin to assume everyone is online. Clay no longer runs meetings without an online component (chat, wiki).

Part 3: Things core to social software

What makes a large, online group successful? After ten years of research, Clay can say with confidence: "It depends." [Laughter]

The normal experience of social software is failure. Yahoo Groups, for example, exhibits a power law: few succeed. But there are about half a dozen things true of software that supports large and long-lived groups:

1. You can't completely separate technical and social issues. E.g., you can't separate the two mailing lists. "You can't specify all social issues in technology." The group can't be programmed. Let the group decide what its value is and give them the tools to defend it; do not try to build the value into the software.

2. In a successful group, there is a core that cares about the integrity of the group and "gardens" it. The software ought to let the group express this fact. If you don't, the group will invent its own ways.

3. Some members need more power than others. E.g., the Wikipedia's "fire brigade" that undoes destructive changes.

What would you design for if you build social software?

1. Anonymity doesn't work in group settings. I need to be able to associate who's talking now with what's been said before. Reputation is not portable from one situation to another: someone who cheats on his wife may not cheat on his taxes. Ebay's linear metrics works well in a linear transaction system but not in non-linear conversation spaces. So, for social software to work, users have to identify themselves and there has to be a penalty for switching handles.

2. You need a way to recognize good members.

3. You need barriers to participation, some segmentation of capabilities. "Otherwise the core group won't have the tools they need to defend themselves." This flies in the face of ease of use, but focusing on ease of use looks at it from the individual, not the group. "The user of social software is the group, and ease of use ought to be for the group, not the user."

4. You have to find a way spare the group from scale. "Scale kills conversations." Metcalfe's Law means the density of conversation falls off rapidly. You need "soft forking." (LiveJournal does a great job of this.) Metafilter shuts off the "new user" page when they get too many users.

"The act of writing social software is more like the work of an economist or social scientist." The people using the software will act as if they have rights. The site is theirs.

In response to a question: Old social software has been architectural: build a place where people can meet. Now we're moving to a ship-building model: Build a place where people can go somewhere together.

[Great presentation.]

FUN QUOTE 1: "Learning from experience is the worst possible way of learning. It's one up from remembering." It's better to learn from reading.

FUN QUOTE 2: Sophisticated computer graphical worlds look real "if you're drunk and in the barrel" looking out the bung hole.

Two New Blogs of Note

Esther Dyson is blogging!

And so is Geoff Cohen, my town-mate and recently of the Center for Business Innovation.

[ETech] Macromedia

Kevin Lynch of Macromedia, a Platinum Sponsor of the conference, is showing an alpha compile of "Central." It seems to make it easy to tie together apps. The first example is of a directory that pulls together info from a variety of sources. It looks a lot like a demo of a portal product but I'm sure it's much more exciting than that. I just don't know why. Maybe it's the way the various apps (e.g., directory listing and the technorati link cosmos for the person's blog) communicate and update. Via web services?

In another example, clicking on an entry in a directory pulls up an office map with the person's sube marked. I'm not seeing what this is an interesting example of.

Developers can build Central apps and Central manages the distribution and fees.!

Aha! Tim O'Reilly asks the first question which is basically a statement about why Central matters: It's like Sherlock on the Mac OS X in that it gives a new way of browsing. It combines the nearly ubiquitous Flash with Web services to treat the Web as a collection of re-usable apps and data, played in a rich client. Thanks, Tim!

Question: If you're trying to replace the Web with Flash, what do you do about accessibility for the blind? And isn't this bolted on top of an animation engine [and thus is inefficient]?

Answer: Yes, it is bolted on top. But performance is sufficient. And it runs across all the platforms on which Flash runs. As far as accessibility goes: Macromedia tries to be a leader, with accessibility support for Dreamweaver. Flash supports hooks into the accessibility features of the operating system.

Question: I feel like we've rediscovered client/server.

Question: Why not just do it in Javascript and SVG?

Answer: You could, but they weren't designed for building apps, and with Center you don't have to worry about a cross platform client.

Question: When are you going to improve the integrated development environment for Flash? We're tired ot calculating pixel widths in order to resize windows.

Answer: We're working on it.

[I guess I'm still bothered by the prospect of more and more of the Web being moved onto a proprietary platform.]

This just showed up from "Chris" on the discussion for this entry. I found it helpful:

It's interesting because of the scale of the userbase (and it autoinstalls), and the fact that it will be everywhere - computers, cell phones, any connected appliance. One of the really neat features is the auto-updating, and working when not in Internet coverage. For cell phones, this is the most important aspect for apps, and one that each developer has to solve themself. Having a central mediator for all this potential traffic is useful.

[ETech] Thursday: Alan Kay Keynote

He's going to complain and then show us some ideas.

He says the past twenty years of computing have been boring because it's been focused on business. The predicted coupling of computers and the human brain hasn't revolutionized human thought in general, even though it has happened in science. That's because we generally are "instrumental reasoners," i.e., we think about stuff only in relation to our current "goal structure."

He shows a great video of the program Sketchpad which in 1963 was the first object-oriented graphics system: A room-sized half-MIPS computer that let you draw shapes on a screen. Totally cool. So early and so dead-on right: it is such a pristine example of the power of object-oriented graphics.

He shows a video from 1968 of Engelbart demoing what looks a hell of a lot like a personal computer. It includes communicating via video and audio with a remote worker. And another shows a 1968 graphical workflow designer. And, he asks, why don't we have anything like Papert's tools for teaching kids how to program? Kay is making his point that we haven't really come as far as we think. We're not making the breakthroughs we once did. He says it's because "We're not even thinking about such things." [Or is it simply that the rate of innovation is always faster at the beginning? And perhaps we haven't done some of these things because there's no actual desire for them? E.g., it may seem obvious to Kay that kids want to and need to program, but he's an uber-technoid.

Noiw he's showing a paint program for kids that looks a lot like an instance of Squeak programming environment. (Yup. I got something right!) It's a slick and easy programming environment: to get the output of a steering wheel he just drew to steer the car he just drew, he drags a field from the wheel's property sheet onto the directional field on the car's property sheet. He gets applause.

Now he shows work by 11-12 year olds using the environment. Some elegant solutions to problems like getting a clownfish to find its meal by seeking the darker color gradient or keeping a car-graphic driving in the center of a twisting road-graphic.

Now he and Dave Smith are showing "pre-alpha" software that Kay calls a "broadband collaboration space." Kay refers to Smith as "the slash in TCP/IP." Given that the number of groups is 2ⁿ, how can we enable scalable group collaboration?

Answer: We're looking at a shared 3D landscape. We're seeing Dave's view and Alan's view on separate screens. The landscape is populated with framed photos of friends. You can look through a "portal" into another space.

There's a cool demo of a flag waving in which the ripples are all dynamically computed. Likewise for an underwater world. And he draws a crappy 2D drawing of a fish and it gets turned into a nicely rendered 3D fish swimming in the shared environment. Kay draws a piece of seaweed. It too is rendered and shared.

Key point, he says: Late binding is good. He's surprised that that's not more widely embraced yet.

The audience gives Kay an extended round of applause.

[ETech] Social Software Alliance BOF

(BOF = Birds of a Feather, i.e., an open discussion with the chairs in a circle. This one is packed.)

Peter Kaminski has suggested a place where standards and ideas for social software can be proposed, kicked around, and maybe even implemented in rough and ready ways. The idea is to have a central coordination point to see if other people are thinking about the same thing. Goal: interoperability. He's set up a wiki for it.

The meeting has gotten off to a bad start by too many people trying to decide how the email list ought to be forked; besides being a relatively trivial topic, the group has no way of resolving such issues. After about 10 minutes, Dave Sifry says that this is beside the point. The question is, he says,: Are we going to do something or not?

Pete asks how the group should be organized. General answer: Not. Let it emerge.

Discussion of the need for a broader understanding of the social effect of software. [Is social software necessarily democratic?]. Pete says that he can see a developer's pledge coming out of this.

Greg makes an impassioned plea for seizing this remarkable moment in our history. And we should make the medium and small advances we can without waiting to figure out all the big issues.

Initial standards to discuss:

Kevin Marks suggests vote links: adding +1 or -1 to a hyperlink to indicate whether you agree with the object of the link.

Pete brings up Matt and Paolo's topic standard, ENT.

Marc Canter brings up threadsML, near and dear to my heart. And an identity system.

Greg wants to manage images as collections of objects.

Saman Far: RFID

Dave Sifry: digital ID and stuff that connects the Web with "meatspace" (e.g., meetup.org). (Dave says he thinks we ought to start with email addresses as our digital identities.)

(I couldn't hear some of the responses.)

A tentative meeting if only because most of the attendees are reluctant to impose order on an emerging social network, but worthwhile. Social software is going to be an important category.

Jason has a good comment on social software...

[ETech] Bookmobile

Lisa Rein shows a brief video of the Bookmobile: a van with access to tons of books and a bunch of printers. Brewster Kahle says that the Library of Congress says that it costs $2.00 to check a book out and back in; he can print a copy of a book for less than that and charges $1/book.

Brewster says that traditionally we've been good about aggregating books in libraries. To preserve them, he's digitizing and replicating. E.g., his group donated a scanner to Egypt and they're scanning 2,000 pages a day.

There are about 16M books in the public domain in the Us libraries. There are about 8M from before 1923 and 8M from 1923-1963 that are in the public domain. About 20,000 have been digitized and accessible. With Creative Commons and Research Library Group, Brewster's group is going to catalog the US reserach libraries to find out what we have and what's out of copyright.

He talks six of his heroes, people who are putting themselves on line for the public domain:

- Rick Prelinger: digitizing film for free use
- Michael Hart: Project Gutenberg. 7,000 books keyed in so far.
- Gretchen Phillips: Digitizing children's books
- Charles Franks: Distributed Proof Reading project
- The Million Books Project in India
- Tim O'Reilly: copyright of 14 yrs

Three things work against the public domain: Copyright, Access and License.

Copyright we know about: we've gone from a copyright of 28 years to life + 70.

Access: The public domain is locked up in private collections and libraries. It's just hard to get ahold of the material. He warns that some companies that should know better are going to be announcing that they're going to assert new rights over public domain materials, making it hard to spider them and access them. [Adobe?]

Corbis (owned by B. Gates) digitized a lot of photos from the National Archive and now claims that have the right to protect it via license. You can go an redigitize it, but getting access is difficult.

We need to bring public access to the public domain. It'll take some work, some technology and some page-turning. It'd cost about $26M (a dollar a book) and the Library of Congress' budget is about half a billion. At least let's do the easy stuff: the public domain.

Terrific presentation. It's great to hear someone who is doing good. And the book samples he passed around, fresh from the BookMobile press, are elegant and readable and cool.

[ETech] GNU Radio

This session's topic — software-defined radios — is an important topic. Matt Ettus and Eric Blossom are the presenters.

It's an important topic because we have a broadcast system based on the idea that the receivers of signals can only do one thing with the signal: turn it back into sound. But if a radio is programmable, we caa\n be much smarter about how we transmit information and we can do much more interesting things with the signal once the radio receives it.

The HD TV demo looked fine, although there were some "motion artifacts" caused by the player they're using. You can play TV through your TV already, of course, but these guys are doing it without a dedicated tuner and HDTV card. And it's all open source.

Says Eric: The politics are hairy. We're waiting to see what the FCC does about the broadcast flag. There's also a "preposterous" proposal that all analog-to-digital devices should shut down if they detect a watermark indicating protected content. But in general, the FCC has been good about software-defined radio; they see it as something that may redefine radio (says Eric).

Even the cheapest computers these days (e.g., 1gH PIII) is more than enough. They are targeting $300-$400 for the board that will turn your computer into a software radio.

[Eric, in response to a question, talks about the myth of interference and new studies of mesh networks showing that smart transmitters and receivers actually increase capacity. May I reference my article in Salon that explicates David Reed's view on this? No? Sorry, too late.]

[ETech] O'Reilly Radar

This is a hastily-assembled session about what Tim O'Reilly and his pals see coming over the horizon. The room is crowded.

Rob Frederick, Amazon Web Services. "We consider Amazon.com to be a technology platform. Developers can use Amazon's backend systems, get product information and customer comments, and create and innovate. Available through their free SDK: search, browse, sales rank, etc. You can use XSLT to pass parameters and Amazon.com will return info as html snippets.

He points to www.simplest-shop.com by a 19-yr-old Rumanian developer who puts his innovations into the open source community. Using his code, you can develop a store front that has features not found on Amazon.com. (Tim points to this as an example of a long term trend: "decomposing" the Net so that we can get data through multiple interfaces.) Another example: Yes.net looks up any song currently playing on a radio station anywhere in the US.

A guy from Bay Area Wide Inernet Access is talking. The demand for wifi is being driven by the inability of the infrastructure to do the last mile. Wifi apparently works real good.

Bunnie Huang is a lonely hardware guy here. "It's physics, solder and assembler." A guy built a heart machine doing wave forms out of a Game Boy Advanced. Bunnie is also working with Field Programmable Gate Arrays.

Andrew Phelps (a prof at RIT) is talking about Phank. It inserts its own additions and hacks into massively multiplier online games. E.g., if you meet someone new in Everquest, you can see if anyone else in your guild has met them. A real-time paging system can let you know when, for example, a dragon is about to be slain.. Phank supports its development efforts by selling t-shirts online. They've built up characters in the game and sold them on eBay for a couple of thousand dollars.

Pretty interesting session although I was hoping to be more amazed.

[ETech] Biological Computing

Eric Bonabeau is talking about "Inspiration from nature for designing computing systems."

He begins with an exceptionally clear example of how ants "solve" the problem of finding the shortest route to a sugar cube. Those that discover the shortest path deposit the freshest pheromones. He now applies this to the Salesman Problem (shortest routes among multiple cities). [It involved math so I played Minesweeper until the scary symbols went away.]

He shows how this form of emergent programming can result in more efficient routing of packets.

Now he's onto the "second lesson from nature: simple rules rule." With ants, a small ant carrying a load will give up that load when it comes across a larger ant. This is apparently quite efficient as proven at CVS drugstores where workers were ranked in terms of productivity and organized into ant-like "bucket brigades." [How appealing!]

Eric asks us to imagine a game in which we are assigned an aggressor and a protector. Each person has to try to put his protector between himself and his aggressor. The overall pattern of behavior is very hard to predict. Change the rules a little and you get a different outcome. And individuals' explanations don't really account for the way it turns out: people don't know what they're part of. But you can simulate this via bottom up modeling, and you can affect behavior through minor adjustments of the rules. E.g., SW Airlines did this for cargo routing and saved $10M a year with a 71% improvement at the biggest hubs.

Eric tells a cuationary tale about blindly following simple rules. For example, army ants may sometimes form a circular path and follow each other around, putting down more and more pheromone that causes them to march ever faster, until they die. [Ah, a metaphor for life.]

Lesson #3: No one has to be in control. His example: Ants randomly position themselves around a big item until by chance they are all pushing in the same direction. Also, nest construction by wasps.

How do we shape emergence? Great demo at the end: Give people a slip of paper saying who they like and who they hate. They are to move towards those they like and away from those they hate. With random initial conditions, we get the same pattern: a rotating line within a following circle. Eric ran the simulation and it was, well, cool.

Practical results: Self-organized satellite deployment, swarm-based senor networks, self-healing networks. Social control and world domination. Coming soon.

Glenn Fleishman asks: How do you model perversity, i.e., "particles" that purposefully don't follow the rules. Eric says this asks what you do if the particles don't like your rules. Answer: you can make the rules robust against perturbation. [Doesn't this mean that you can make rules that result in particular behaviors that the particles — we — don't recognize as bringing about that group behavior and that we don't even want to follow?]

[Would someone like to tell me if this shows the importance of Wolfram or his irrelevance? Or neither, of course.]

FUN QUOTE: You never find mergers in nature. Only de-mergers.

[ETech] Conference Blog

Apparently the conference blog aggregator is a list of trackback links.

Role of Technologists in Politics

The first two sessions of the Emerging Tech conference raised the issue: What should technologists be doing to keep the Net free and content open?

Here's my answer, as profound as it is detailed:

First, technologists should make technology that supports openness.

Second, technologists should explain that technology as clearly as they can.

Here's an example.

Right now, we're in a battle over how controlled content can and should be. Everyone (?) agrees that creators ought to be compensated for their efforts. The question is: Is it necessary, fair and good for creators to always be compensated, in a one-to-one way, for every encounter with their works? Should we shut down photocopiers because sometimes they are used to violate copyright? Should we shut down VCRs for the same reason? Should we prevent people from lending books to their friends? Should I have to pay the author again if I choose to reread her book? Nah. But somehow the entertainment industry has persuaded Congress that any uncompensated use constitutes piracy. Technologists ought to convince Congress that the attempt to lock down all usage is either impossible or carries with it such terrible side effects that it is undesirable.

And then they ought to hack the control mechanisms.

[ETech] Wednesday AM: DRM panel

Dan Gillmor is moderating a session on Digital Restrictions Management.

Joe Kraus of DigitalConsumer.com: Congress is thoroughly convinced that DRM is about ending theft. They do not see it at all as about openness, access and fair use. This is largely because Hollywood has been effective whereas Silicon Valley believes that if its recitation of facts didn't work, it needs to re-state the same facts.

Wendy Seltzer of the Electronic Frontier Foundation. The law is being interpreted to mean that when you buy a DVD, you haven't bought the right to use it as you please but only in DVD players acceptable to the content creators. The word "theft" shuts down all discussion. [What's the counter?]

Bunnie Huang who did the X-Box hack is here as a DMCA victim. He hacked the XBox. He asks: After we undergo the massive hardware change required by Palladium, will we feel safer? Couldn't we do it just as well today using technologies like PGP? Huang says that Microsoft says they're not worried about the hardware hacks that could undo Palladium, e.g., something that goes in the memory slot to unprotect it. [Keep in mind that we're being protected from ourselves.]

Cory Doctorow says that we've heard from nerd-determinists and nerd-fatalists. He says that the good news is that Napster built the largest collection of human creativity ever and did it totally bottom up. When the copyright law was used to "burn that library to the ground," the library rebuilt itself. The bad news is that the problem doesn't lie solely with Congress, the recording industry or Silicon Valley. It's our fault: there were 57 million Napster users. That's more than the number of votes W got in 2000. The real point is that copyright's purpose is to build libraries, and its tactic is to compensate artists. Napster and Kazaa don't have models for that and it's a real problem. DRM is the answer to a question we shouldn't be asking: How do we burn the library burned down for good? The real question is how do we come to some compromise by which there's fair compensation. The "broadcast flag" isn't a compromise; it gives the entertainment industry a veto over PC design.

Cory says: The next time someone says "We have to stop Internet privacy," you should reply: "Rather than burning the library, let's talk about how we can compensate artists." [But the entertainment industry says that they are asking that question. They say that in order to enable compensation we have to turn off the free spigot, which means altering PCs, passing laws, etc. I think the compromise isn't over the fact of compensation but over accepting that there will inevitably be some use-without-payment, just as there is when we record off the radio, make a back up of a VHS, etc.]

Joe: We've lost the debate on theft. The real debate is really over incumbents vs. innovators. We should stop talking about music and movies and begin talking about what we know about: technical innovation where investors are being sued for investing in Napster, etc.

Q: Will the inclusion of DRM in Office 11 give people a positive taste for it? The panel disagrees. [How about a Creative Commons license stamper for Office?]

FUN QUOTE. Cory: "The compensation for science fiction writing isn't small, it's historic, it's quaint." (approximate)

Where's the ETech blog?

If you know where the O'Reilly Emerging Tech conference blog aggregation page is? If so, please send me email: self@evident.com. Thanks.

[ETech] Opening Session: Howard Rheingold

Rather than rehashing his excellent Smart Mobs, Howard is urging this group of techheads to keep the Net free and use it to spread freedom to the real world. We should be users, not consumers, he says. End to End ought to be maintained. This group in particular should understand the regulatory environment. The technoids will be more effective by building new technologies rather than attempting to fight against the professional politicos. We need to create a "preserve" where we can invent the next generation; the preserve should include opening spectrum for more innovation.

He asks: Why not invent technology that lets us deal directly with musicians rather than having to go through the recording industry? [I don't think that this is a technological issue so much as an economic one.]

Build platforms for self-organizing networks. Watch what happens with trust.

Design Principles: Support End to End and link to others.

In the Q&A, Howard says we need to get the defaults right when it comes to privacy. "Only geeks play with defaults."

I ask: Since the default is about to flipped from anonymity to identity, what about digital ID? Howard: We need to know who we're talking with although not necessarily tie it back to a real world identity. Cory follows up: Since these pseudonymous identities may represent groups or bots, what does that do to trust? Howard replies that so long as they act in a trustworthy way, we don't need more ID than that.

[When I say "we," I mean "the technically adept people here," i.e., not me.]

Help the Burning Bird

From Jonathon Delacour, about Burning Bird:

As you are probably aware, Shelley has had a tough time financially over the past year and has recently announced the possibility that — because she cannot currently afford hosting — she may be forced to discontinue her weblog.

To ensure that Shelley can continue to blog, I have established a Keep the Bird Burning fund, with donations payable via PayPal. The full details are here.

Voting Standards

From Stanley Klein:

There is an ongoing effort in IEEE to develop standards for voting systems. The project is under Standards Coordinating Committee 38. The web page is at http://grouper.ieee.org/groups/scc38/ There are two projects there. The main voting system standards project is P1583.

... I have long been concerned that the bulk of participation has been by representatives of voting machine manufacturers. ...

This effort definitely needs increased participation by people who are expert in information security, interested in voting systems, and not part of the manufacturing community. As a minimum, it needs participation in the voting pool when the standard comes up for approval in the IEEE Standards Association.

Time is short. The next meeting is scheduled for May 19 and 20 in Columbia, MD. ... There are email lists for which the archives are posted on the web site. The drafts are also posted there.

Please either get involved yourselves or urge colleagues to get involved...

[ETECH] Opening session

At the O'Reilly Emerging Tech conference.

First things first: Great wifi. They're even renting wifi cards, although you gotta think that this is the last ETech conference they'll expect they'll need to rent wifi cards at.

It looks lilke there are 500-600 people here. Howard Rheingold is about to speak...

Literary bloggers

The discussion of authenticity and self and blogging leads me to wonder once again why no one is writing a novel in blog form. Or are they?

And which literary characters do you think would make good bloggers?

Roskolnikov

Ishmael

Emma Bovary

Cyrano de Bergerac

Mercutio but not Romeo

Huck Finn but not Tom Sawyer

Hester Prynne but not Arthur Dimmesdale

Daisy but not Gatsby

Blofeld but not James Bond

Spider-man but not Superman

Steve Himmer replies:

Aside from novels being presented via blogging software (such as Alex Golub's and my own, there are actually a number of projects using the blog format as their basis. Some are listed here, and there's also Plan B.

The best blognovels though (whether a blog can actually be a 'novel' or is necessarily a different form... I'll leave that question alone for now) might be the ones that don't reveal themselves as such, I think.

Thanks for the pointers, Steve. I actually meant to refer to people writing blogs that are fictitious (and not in any fancy POMO way) and that over time add up to what we would recognize as a novel.

Tom's Easy Little Questions

Tom asks three questions. Simple little things, really. Oy gevalt!

Here are stabs at answers to two of them:

First, Tom draws a contrast between my assertion that our Web selves carry on their lives purely in the public of the Web and Doc's recent blog about the World Live Web as connected to the real world. Tom asks how we can ever know anything reliable about a RW self based on the Web self. Well, that depends what "reliable" means. If it means reliable enough for a transaction, then a credit card number suffices (and a digital ID is overkill). But that's not what Tom means. If "reliable" means what it does in RW friendships, then I guess you have to meet the person f2f. (And, in a fit of pointless cleverness, let me do the ol' switcheroo: How can our knowledge of a RW self tell us anything about a Web self? Or does the fact that that question makes little sense mean that we think the Real self is the Real World self?)

Third, I suggest that the questions of authenticity, etc., arising about weblogs apply to any narrative, not just to weblogs. Tom replies by asking what's specific about weblogs in general. "But that's another post," he promises. I look forward to it. But there is something specific about weblogs that make questions about self and truthfulness appropriate: weblogs are in fact public selves...which is where my initial post started.

(Tom also corrects a link in one of my previous blogs. I've back-fixed it. Thanks, Tom.)

Michael Isn't Friendsterly Anymore

Michael O'Connor Clarke explains why he's quit Friendster. Well put.

Here's what makes me queasy about the place. I'll be joining Michael in non-Friendster land soon. I'm hanging in experimental-like just to see if anything develops. Also, I want to use it as an example during my talk at the O'Reilly Emerging Tech Conf I'm flying to tonight.

For those who are keeping track, I'm sick as a dog: sore throat, sleepy, even a touch of fever (although my wife — famous for her warm hands during my illnesses — denies it). I am not looking forward to flying transcontinentally tonight, and I doubt the people inhaling my toxic fumes will be happy about it either.

Madonna Anti-Download Prank Backfires

From the UK's unreliable tabloid, The Sun:

By SIMON WHEELER A bid by Madonna to trick fans who download music from the internet without paying backfired - when a hacker caused mayhem on her website.

All 11 tracks from her new album American Life appeared to be posted on Madonna.com for downloading. But when the files were played back listeners heard Madonna saying: "What the f*** do you think you're doing?"

One angry computer buff got revenge by hacking into the website and adding his own message, saying: "This is what the f*** I think I'm doing."

He then left links to a site where the songs really could be downloaded for free.

Madonna.com was taken down for several hours yesterday so the message and links could be removed.

(Thanks, Greg, for the link.)

Self, Truth and Lies

As you probably already know, there's a fascinating thread about authenticity and truth in the selves we're constructing via weblogs. This piece, late in the thread, by Burning Bird is a good place to start. And this piece by Jonathon Delacour is seminal.

I love this topic but I don't see what's specific to weblogging about it. Don't the same questions apply whenever we talk? I have never told an anecdote or story that wasn't fictitious in some sense. Except on the Web, our self is purely public and written, so we can't fall back on the myth of the Inner Private Real person that allows us to act as if there's the possibility of our "outer expressions" corresponding to our Inner Real Self. I.e., the false possibility of authenticity is closed off to us in the virtual world. (I'm burning to say more but I have to go out to a meeting. Ack!)

French is just another word for nothing left to lose

Now that Frank Paynter has changed his name to "Freedom Paynter" because "Frank sounds too French somehow," and the NeoReactionary Republicans are accusing the Right-Wing Republicans of being "Franco-Republicans," I feel impelled to take a stand and say:

Not only is the country beautiful and the art is beautiful and the food is beautiful, but the people are friendly and warm and I agree with France's position on the Iraqi war: If this was a war about WMDs, we should have given the inspectors more time.

So, to protest the "French" to "Freedom" translation, I am going to start referring to the Republicans as the "Gaullist Party."

PS: Frank was joking.

Group Bayesianism

I blogged the other day about a Bayesian spam filter that runs as a service to which you can hook up your local Pop email client. Bayesian filters learn from their analysis of the word usage patterns of content that a user has initially sorted into spam and non-spam piles. This new service, Death2Spam, pools the sortings of all of its users. Since I am a very happy user of a Bayesian filter (PopFile) that runs on my own machine, I asked the service's creator, Richard Jowsey, why we should prefer a service like his that pools all subscribers' spam sortings.

Q: What do you about the "my spam is your important msg" problem? Shouldn't a well-trained personal collection be more useful than a large multi-person collection?

A: Damn fine question, sir! Yes, and no. Our generic database is currently 99% accurate for the average user. It contains word-probability analysis from about 100k messages. And does the basic job really well, for most people. BTW, all our server stats are freely available to worthy causes like anti-spam research. Interesting reading.

We're presently adding a "premium" (aka geek) version, which will allow you to combine personal database probs with the generic data, weighted per your preferences. Plus support for adjusting upper and lower unsure limits. Plus a URL crawler for nailing micro-spams. Plus the mandatory (but largely vestigial) black/white lists. That ought to keep everyone happy... :)

Micro-spam? I don't know what that is but I have a bunch of offers for extending it, thickening it and making it last longer if that'll help any.

FWIW, After a few months of using Popfile, occasionally correcting its mistakes, it's now about 99.5% accurate. That is, about 1 in 200 messages that it's classified as spam is in fact non-spam.

Ambiguous Signs of War

Our daughter Leah has noticed two war signs that, depending on where you place commas, are completely ambiguous about their position:

Mark Lombardi

Valdis Krebs, who works on the automated mapping of social networks, points out that the late Mark Lombardi's astounding, hand-drawn maps of political connections have been gathered into a traveling exhibition. As Valdis points out, even the titles of the works are provocative. You can see some examples here.

Bayesian Filter Service

Richard Jowsey has a new Bayesian spam filter service in beta. It's free for now ("and to worthy causes forever") and will be $39/year after that. Unlike Popfile, which I continue to find an astoundingly good filter, Richard's service doesn't run on your machine but on his servers. I haven't tried it.

Richard's site has a useful list of links to more info about bayesian anti-spamming.