OK - now that we’ve shipped a few networks and I’ve written up how one’s ID, Persona and Groups (#1) and persistent ubiquitous content (#2) are important areas in building out the mesh, I’d now like to focus on an area which has been bubbling up for years and which will also play a key role in the mesh - shared structured content servers (#3).

Call it micro-content, micro-formats, tagging, meta-data, semantic web, shared knowledge bases or what have you - the idea is that content comes along with extra ’stuff’ associated with it and that all that content and data is available on public, shared servers - where a community can contribute to it - and it’s overall value goes up.

Wikipedia in one way - is a shared structured content server. So is DMOZ. And there are all sorts of knowledge bases, open courseware and shared resources available on the web which could be considered ‘shared structured content servers‘.

But what I’m talking about here is one step beyond that. Something a bit more specific. It’s about shared Events, Reviews, local ads and info, maps, digital assets, even people databases or aggregated groups. Having APIs that can specify “Walnut Creek, April 20th, 2008 at 4:20 PM” kind of accuracy and granularity.

That means that the service needs Open APIs so developers (like us) can access it and rely upon it as infrastructure. Structured Content as infrastructure.

One way to think about this - is that just as Brad’s Thoughts called for shared servers filled with your social graph of people, so am I calling for shared servers of ALL kinds of content - media posts, events, reviews, tags, etc.

Let me start off this post with a little history.

In a moment of inspiration, I wrote up a blog post back in Feb ‘03 about me and Joi Ito having an imaginary dinner at a restaurant in Tokyo. In this post (sorry most of the image links are broken) I ranted on about techniques for building the mesh and shared micro-content servers - where one could post movie, restaurant or new camera reviews for all to see and benefit from. This notion tied in mobile devcs - so that one could post a review of a movie they had just seen or restaurant they had just finished dinner at - and get those reviews onto shared servers - ASAP. Well now its five years later - and I’m still ranting on about these ideas.

By having structured content servers as infrastructure - whole new kinds of apps and services can flourish - from mobile, the car, the living room - even from our PCs and Macs. What exactly we do with all this micro-content (or structured content) has been speculated on - since T B-L first wrote about the semantic web - way back in 1997. So all I’m doing here is trying to tie all this idealism into building the mesh - today.

Microformats were developed a few years back as a way to tag and delineate HTML pages so that the structure of the content in the page could be exposed to the rest of the world. Microformats ingeniously used the existing HTML tag format - as a way to bring the semantic web to reality - without waiting for an entire re-casting of the web in rdf.

That led to hCard and XFN becoming major formats in the ID domain. hReviews are now a major way to find reviews and hCal (and iCal) are standards surrounding events and calendar appointments.

We (BBM) helped start an effort called structuredblogging which was to take microformats and other micro-content related standards (for feeds and files) and offer plug-in tools for Wordpress and Moveable Type - which would enable folks to easily produce Reviews, Podcasts, Events, Vlog posts, etc.

That code is still available - here.

Google has shown some interest in this area - with something called Google Base. And other platforms and tools have been created that were explicitly designed for enabling folks to create meta-data associated content (unfortunately many of these tools have gone away.) That’s because meta-data and structured content have not really “caught on” (eg. its not some big trend like Twittering or FriendFeeding.)

But that doesn’t mean that having meta-data along with data isn’t extremely important and I still mourn the fact that the world of podcasting got launched without clear meta-data standards (so we could search for “who’s on this podcast” or “what’s this podcast about?“)

Twine and Freebase are two recent platforms which fall squarely into this domain and I’d like to point to them as great personifications of my ideas about this domain. Both Freebase and Twine store info that user’s plug in about just about anything and they enable users to associate various kinds of meta-data or info to that stuff being plugged in.

They both offer APIs and wish to become part of the mesh’s infrastructure. These kind of repository platforms are totally coolio because they assume the mesh will exist - they assume that expertise and specialization is needed in this domain and they’ve grabbed the bull by the horns and gone ahead and BUILT some shared structured content servers!

Now imagine if we had this same sort of shared, public repository for just Events. Imagine aggregating and combining EVDB (Eventful), ZVents, Renkoo, WhizSpark, Acteva, Eventbrite and any other sort of event or concert or live happening web service into one publicly shared repository? With Open APIs available so any other service could tap into this repository and treat it like infrastructure. This is the kind of shared server which I can ping and ask “what’s going on in Walnut Creek on 4/20 @ 4:20?”

Open APIs on shared event repositories - will be a key foundation for the mesh.

The same thing goes with shared servers filled up with Reviews.

Now let me tell you a little story.

Way back in the early 90’s I was shown AT&T’s real-time MPEG cards streaming multiple (I think it was 3) channels of MPEG video - off of the same card - at the same time. They had built an ‘on-demand’ video system around these cards and were going on and on about the future.

When I clicked on the movie selector, I saw Citizen Kane (the greatest movie ever created) and was horrified to see this 2 sentence review associated with it: “the tale of an overweight movie mogul. Watch his rise and fall….”

When I asked them who wrote these reviews, they talked about costs of data entry and how they had hired a firm in Kansas to create 1,000’s of reviews - really cheaply. What I realized then was that shared servers of movie reviews would have to come into being, if for no other reason to aggregate all the GREAT reviews of movies - in one place (I always remember the famous Pauline Kael essay on Citizen Kane.) Think of how many college students, film buffs and movie nerds there are out there - who could fill up these servers will all sorts of great stuff.

Why should every cable system, on-demand system, DVD rental business, etc. have to create reviews for the same dam movies over and over again? Can’t we aggregate ALL the reviews for ALL the movies, songs, albums, shows, operas, plays, etc. - onto one server? And provide APIs to these databases - so we can have structured content as infrastructures?

And what about tags? Can’t we store all our tags on shared servers and let others use them? And not only the tags themselves, but the links to what the tags are associated with. That’s kind of what Technorati tags, delicious and Magnolia are all about - but we need to promote this capability within the context of content as infrastructure.

Maintaining open shared servers is a tricky business and needs to stand up to spam, hackers, politics and vendor domination - but it CAN happen!

What the mesh has to be built with - is the combination of these sort of shared open services and servers, along with platforms and applications which are designed to make money. This kind of hybrid combo of proprietary APIs with open APIs will become the infrastructure on which we’re building the mesh.

That way - no one vendor or entity can control this crucial aspect of the mesh’s infrastructure. I sure hope Microsoft and Google understand that! I’d hate to see a battle erupt over different meshes - cast in different formats - just to fuck over the other mesh!

Its not like Microsoft is gonna give up controlling their APIs. They’ll let folks use them for no charge, but believe me on this one - Microsoft’s APIs (and Google’s for that matter) will always be proprietary. And so will Eventful’s, Yelp’s and the IMDB’s .

But we’ll have Music Brainz and Phil Pearson’s ‘New Zealand coffee review’ database to show how to do it right. We (Broadband Mechanics) will propose a kind of open server (called OutputThis) which will store various destinations which bloggers wish to maintain - so they can update their destination list in one place and have multiple tools use that list (this would work well with SixApart’s recent Blog It feature.)

And if for some reason we end up with multiple shared servers which have incompatible schemas and data formats of reviews - then we can simply map one format into other schema - to attain compatibility.

Technically we’re really just talking about shared XML servers and open knowledge bases. But it takes us marketing dweebs (like me) to view this as an entirely separate ‘domain’ of the mesh.

Now I bet a few of you who are paying attention are thinking “Hmmm - didn’t these same kind of ‘persistent’ content ideas get featured in Marc’s LAST blog post (#2?)”

So now you’re starting to see the incestuous and overlapping nature of the mesh!

Things like shared Events servers can be thought of as BOTH persistent ubquitous content AS WELL as tagged, indexed and sorted public shared servers. Not all persistent content has to have extra meta-data attached to it (though it should of course) while most shared content cannot (legally) be copyrighted materials.

There are certain constructs, like tagging, and principles like Interoperability or Dataportability - which are constant across ALL the areas of the mesh. This is what makes laying area charts on top of each other - so much fun!

Tags can unite us together. ‘Reggae‘ tags can send me to events in Jamaica (or Brooklyn), hats on eBay, music on eMusic, videos on YouTube or movie reviews on ‘Ain’t it cool News’. Once we have the mesh - all of these domains will freely interact and inter-connect between each other.

Interoperability is a constant - being able to talk to, create actions and verbs, actually DO something between networks and worlds - is a crucial aspect of the mesh. As is Dataportability. Users need to control their data - while at the same time vendors have to see the BENEFITS of making users happy!

I’m writing up these areas as separate posts - so we can get a handle on all the dimensions, playing fields and politics involved in getting all this stuff to ‘mesh together’.

So here’s a super simple area chart showing a user connecting into all sorts of shared structured content servers, utilizing tagging to connect to other areas and dimensions.

Action Items in this domain include:

- shared structured content servers - who’s day has come. NOW’s the time to start creating these services.

- APIs for content infrastructure - two-way APIs to suck content out and utilize it - while at teh same time contribute content as well

- standards for content meta-data - yes I know I’m asking the impossible and there have been TONS of attempts at establishing standard taxonomies and ways of organizing meta-data. But that doesn’t mean we can’t keep trying.

- new kinds of services which utilize structured content - this business is about showing examples of what can be done. We need some great demos!

Action Items in general include:

- make sure that ALL the open standards continue to rise in popularity, that all implementations are compatible with each other and that more open standards get created. And if there ARE any incompatibilities - simply map between them. Don’t fight about it - life is too short and your family would like to have some time with you!

- testing and compatibility labs - a place where we can guarantee that everything works together. Building a COMPATIBLE mesh will be a challenge - and it won’t ever happen if things break or don’t work.

- two-way APIs - until we can write back into systems and services as easily as we can get data from those services we won’t have a symmetrical architecture and a successful mesh environment

- establish OutputThis as a standard for content producers to list all of the destinations they’d like to route their content - to. SEE Dataportability.

- establish opt in controls - so uysers can control if someone else can export their data - somewhere else.

Summary of persistent data repositories discussed here:

Freebase, Twine, Wikipedia, DMOZ, Technorati, Zvents, Eventful, Renkoo, Yelp, Muxtape

Major players and people to watch and listen to:

Phil Pearson, Tantek Celik, Chris Messina, Dan Brickly, Mary Hodder, Jon Husband, Keith Teare, David Weinberger, Stephen Downes, B.K. DeLong, Brian Dear, Marc Eisenstadt, Paolo Valdemarin

Major organizations and advocacy groups:

Microformts, StructuredBlogging, FOAF, W3C, Internet Archive, ourmedia, the Higgins project

Final NOTE: This is my third post in this series - I ain’t done yet! Next up - #4 - the Live Web

This entry was posted on Wednesday, April 16th, 2008 at 4:35 pm and is filed under Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.