InformationDynamics

Actually, it was James Surowiecki’s book (Wisdom of Crowds) that got me thinking about this whole Beyond Folksonomies panel. Surowiecki will also be speaking at this year’s SXSW. I see that co-panelist Mary Hodder has summarized a recent presentation of Surowiecki’s. (In addition, Mary was highlighted on the SXSW page. Her comments on microformats tie back in to my understanding of this meaning-making on the web as well).

The crowd that I would like to tap the wisdom of is none other than the SXSW attendees. The potential for cross-fertilization is enormous in this gathering. One of the ways that we will encourage that is by organizing a gathering for dinner after our presentation (avoiding conflicts with parties) so that interested folks can carry on the conversation. That information will be posted on this site. In addition, if there is other follow up information during or after the conference, this will be the site to find it.

Wouldn’t it be great if every panel had a similar page? Why can’t it. What would happen if we tapped into a the synergy available in this particularly bright crowd?

Let’s find out!

Facts, by themselves, mean nothing. They are in need of a context in which to interpret them.

In the theory half of the exploration of Hawkins’ book, I conclude by asserting that memes are facts that combine lower level memes and an encompassing context to convey greater meaning. As complicated as that may sound, it just happens naturally in our brains. We don’t have to think about how to wire up our memes to convey meaning. But, if we want to build intelligent machines, it may be necessary to bring a bit of conscious intention to the process.

Currently, taxonomic classification is pretty arcane process to the average net user. Yet each person unconsciously performs folksonomic-like classifications without even thinking about it. How hard would it be to leverage the existing activity of hundreds of millions of net users to build higher-order memes? Probably really hard if one is trying to get the average user to be completely interchangeable with any other net user. But if we allow each person to flourish by striving to make sense of information uniquely interesting to them in their own way, I think the process could be greatly simplified.

If those skilled in taxonomic specialization spent time creating middle-layer meme-maps, and leading-edge users started wiring from the bottom-up by hooking folksonomies to these middle-layer meme-maps, I think the pioneering effort would encourage others to follow, and the collective intelligence would start to wire itself together naturally.

Practical applications: Have you ever had a great idea for a book, movie or software program? If so, you likely never did anything about it because you couldn’t find a person who would actually give your idea the time of day. I imagine that a lot of good material is left undiscovered in the haystack because the filter has to be incredibly conservative based on the bandwidth of the top nodes (the publishers, film producers, or venture capitalists). By wiring up the meme-layers in between the bottom and top layers, worthwhile ideas have a better chance of being noticed.

We already use collaborative filtering in a very crude way, and what we get for it is network television, Hollywood movies, and Microsoft. By extending the utility of collaborative filtering to separation of signal from noise on a broader scale, we can possibly find better solutions for a better personal fit.

The trick is to avoid falling into the “top 10″ trap. The cultural tendancy to find the best of a category ignores the fact that each person’s criteria is going to be different, and so these lists will be of limited use as a global reference. If I’m looking for good, cheap mexican food in my zip code, I’d like to be able to search for people who like other restaurants that I already know I like. Folksonomies are a good step in this direction, but leave too much work to be done by the user. The way I envision this working more easily is by allowing users to help each query along. Rather than using a strictly algorithmic solution, like Google, use a human assisted approach.

First objection to the above: authentication. How does one prevent a person from spoofing, spamming and all the other coercive trash? Look at my other post on Identity 2.0.

Note that I’m not claiming that the problem is trivial, but I do think that some of the solutions (like current AI approaches) are barking up the wrong tree. I think that we can do better by using the best pattern recognition hardware on the planet: good ol’ fashioned human neo-cortex.

I have only recently read the Jeff Hawkins book, On Intelligence, but it has profoundly influenced my view of the unfolding of consciousness through technology.

Hawkins’ thesis includes two points which I think are particularly worth noting. The first is that the cortex is essentially uniform through-out. The specialized areas of the brain that deal with sight, smell, touch, and so forth, are only conventional. There is no structural or functional difference in these areas, and if the inputs are rewired, the new wiring will work just as well as the old. All areas of the cortex are equally good at interpreting patterns in input signals, irrespective of the source or context of those signals.

The second point is that invariants occur in all six layers of the cortex. The conventional view, according to Hawkins, is that invariants only occurs at the top levels, and this is where meaning comes into play. It is how we recognize cats, dogs and trees, even though no single instance of any of the actual instances of these items looks like an idealized version. Hawkin’s hypothesis is that each layer, even the lowest, works by establishing invariant primatives (the way that I understand this is that these primatives are pre-linguistic and therefore subconscious — we don’t have ready access to them in a meaningful way).
How is this relevent to “Beyond Folksonomies”? My claim is that taxonomies are ways of capturing invariants in a collective intelligence at a top layer of meaning, and that folksonomies are low layer invariants in the collective intelligence (This paper on first glance seems to take the same view - I haven’t read it in detail, but it looks like it is headed in a direction that I like).

This is all a bit too technical for this casual discussion, but it boils down to this point: folksonomies are a necessary foundational layer to making sense of the entirety of human knowledge, and taxonomies are a necessary top-point. What is missing is the connective tissue between these that allow people to natively make linkages between the bottom and the top.

I’m certain that my summary is a butchering of the facts for someone knowledgeable in these matters, but the metaphor of it works pretty well. Memes are those invariants which can travel between brains via language, imagery, etc. Higher layer memes convey greater contextual information and deeper meaning. Meaning at any layer is a function of correlating invariants of the lower levels into a coherent pattern.

My interest in looking beyond folksonomies is rooted in a longstanding desire to attract a critical mass sufficient to generate a wide-spread wisdom-of-crowds effect. Currently, del.icio.us is still way to esoteric for nearly all my friends and relatives, and even a sizable percentage of the developers I know. I can see clearly that collaborative meaning-making will eventually catch on, but the specific application which will gain mainstream appeal is still fuzzy.

In my mind, there are three interdependent issues which conspire to make the transition to the big time a challenge.

1) ATTENTION: Far too many people still leave the job of aggregation and filtering to big media. I think they’re going to need to be lured into sharing their preferences in any real depth. I personally would love to have my attention trail logged implicitly, available for augmentation and publication as I saw fit (how much of myself I reveal, and to whom). If I only trust a small group of people, then only they ever get to see my attention trails. If I could insure this, then I wouldn’t mind implicitly logging my activity (online, or with GPS cell phones, offline as well). Is that really feasible though? Maybe with …

2) IDENTITY 2.0: There is a very interesting podcast that Josh Porter and Alex Barnett had with Dick Hardt (check out his identity presentation, if you haven’t seen it) and Kim Cameron on Identity 2.0. At one point during the conversation, I believe it is Kim that raises the concern of privacy as an objection to widely shared attention metadata. Perhaps I’m reading more into ID 2.0 than is there, but shouldn’t partial or limited authentication be possible? If so, then I could have some say over how the attribution for my attention metadata appears, all the way from completely anonymous to fully public. Partial authentication looks like: “over 40, male, living in Texas”, all authenticated without divulging more than that. Granted, I don’t think ID 2.0 is there yet, but I hope that we’re moving in that direction. There is still another issue…

3) TRUST: I don’t just mean trusting the protocols, or members of the larger public. I really mean trust of the collective itself. The biggest objection that I hear when I talk about Wisdom of Crowds is that group-think or the “stupidity of crowds” can’t be reliably overcome. I don’t think enough people realize that the NY Times derives is credibility from the masses believing in it (not to mention the value of the dollar). As we move deeper into the mainstream acceptance of a web 2.0 culture, I think a larger portion of people will become increasingly aware of, and comfortable with, the power of and wisdom of the collective. But, I think they will need to be led to this awakening gently.

I’ve been thinking about such an application for a while, and as I watch the abundant offerings flow out of the innovative minds of fresh young upstarts, it seems more realistic every day. Yet, it seems far too few, even among the hard-core bleeding edge, are looking that far into the future. I too am wary of vapor-ware nature of pipe-dreams, but I’d like to spend some time thinking about the cross-functionality which will allow culture to become self-aware in a way analogous the sentience of human beings individually.

Folksonomies are a great start, but they only get us part of the way there. This chasm is wider than del.icio.us, or even allpeers or others can accomplish. What are some requirements and feature sets that can be established as points of reference on this new frontier? Is this too ambitious a conversation to have in a one hour panel discussion? If so, how do we wish to scale the presentation back?

BeyondFolksonomies.com is a blog for the panel of the same name presenting at SXSW 2006. Because it would be a little strange for a topic this wide-ranging to be constrained to an hour of the 4 of us on the stage and a few conversations afterward.

So, we’re starting the conversation a few months earlier and will be hanging out here to keep talking during and after as well. Notes, resources, etc. that apply are here as well. So, whether you’re just sitting in the audience at the conference wanting to look up more about us as panel participants or to dig deeper into the topic we’re covering or just stumbled here because you’re interested in folksonomies and meta-data, welcome.