Explicit vs. Implicit Data and the Wisdom of Crowds

Harrison Hoffman has an interesting article up on CNet about the Wisdom of Crowds failing. The phenomena that he's really observing though is a gaming of the system at IMBD. People are voting the Dark Knight movie up and the Godfather down in order to influence the rankings.

Explicit information where people take an action to rate or vote can be wonderful and create fantastic resources. However, it doesn't matter if you're at IMBD or Digg or Yelp it can also be gamed.

At Aggregate Knowledge we have found that Implicit information where systems look at what people are naturally doing tends to be much more reliable.

It's not what you say - it's what you do!

Albert Lai launches Kontagent

My friend Albert Lai just launched his new company Kontangent. It's a viral analytics tool that seems pretty neat! I am going to get the demo soon I hope. Albert is one of the few people who has started more companies than I have!

TechCrunch has the writeup.


Fresh ID gets it wrong

Kristi Colvin at Fresh ID has a rant about recommendation engines that I think really gets it wrong.

Kristi believes that recommendations don't serve users but in fact only serve the retailers or the publishers that use them. How can that be? You can't force someone to buy something just because you show it to them in a recommendation window. The only reason someone buys something is because they ultimately want it.

Kristi - where's your data that users don't like recommendation engines besides your individual subjective experience?

The true measure of how well a recommendation engine works is by seeing how many people find things they want through it, not the subjective measure of whether you think it looks good or not.




What comes after Search?

It's clear that Search has been a dominant force on the Internet and that Google is has won. According to Compete, Google had a 71.5% share in search in May 2008.

Interestingly though - time spent searching is going down. Data from the Online Publishers Association says that time spent searching dropped 15% from June 2006 to June 2007. Time spent was 4.5% in June 2007.

We need to help people the other 95% of the time that they're online!

Friendster hacked?

I got a couple of complaints from people saying that I had been sending them spam messages from Friendster.

Oddly enough I started to get spam from my brother as well:

Friendster I looked in my sent folder and it was empty. My brother's sent folder was empty as well.

I changed my password but it looks like this is widespread...


YouTube gives up user histories and profiles

I just found out through Techmeme that YouTube is being forced to give all their user profiles and history to Viacom by a judge.

This is a massive breach of privacy that Google is being forced into and seems crazy to me. Certainly Viacom could get an understanding of what users were watching without the things like user names being divulged.

Recommendation Systems in Social Media

Muhammad Saleem has done a nice write up of how consumer social media sites are using Collaborative Filtering over on Read/Write Web.

The only thing I disagree with is equating Collaborative Filtering with Online Recommendations. Collaborative Filtering is only one technique that one could use to provide recommendations.CF is a good tool but you have to be careful to apply the right tool to the right situation.

At Aggregate Knowledge we actually use a variety of different algorithms types to determine what someone might be interested in. For example taking into account seasonality, recency/frequency and lexical matches to show you things you're most likely to want to see.

Digg Launches Recommendation Engine Beta

Digg just announced the launch of their recommendation engine. There's a really interesting video with their lead scientist Anton Kast talking about how the technology works on the Digg blog. More coverage here at VentureBeat and TechCrunch.

Digg has been one of the real pioneers in the Discovery space with their use of explicit user data to help surface interesting content.Some of their new approaches are similar to techniques that we have worked on at Aggregate Knowledge and I'm curious to see how well they work. My guess is that they will see a decent sized bump user engagement if you look at pages per user or click through rate on their recommendations.

Anton really nails it when he talks about how some of the biggest problems are doing this in real time and at scale. It shows a great understanding of the real problems in teh space.

If any Digg folks read this I'd love to get an account to check it out and give feedback!



Much Respect

I just found Russ Beattie's blog post where he talks about shutting down his company Mowser.

The post is a really mature analysis of his situation and reminds of when I left the first company that I started. I'm not an expert on his space, or on his company but I have been in his shoes before staring into the abyss.

Russ - best of luck in the future. I hope you'll find that the experience has made you strong and that you do try again someday.

I have a day job

I haven't been posting much lately. A lot of that is related to busyness at work. The two seem to be inversely related.

It is interesting to me this whole meme about blogging can kill you from the NY Times. It made clear to me the very big gulf between professional bloggers and those of us like me who are just messing around with it. If I was a pro-blogger and very busy then you would see my posting volume increase.

I don't think that you have to make the majority of your income from blogging in order to be a pro-blogger like Mike Arrington or Robert Scoble but certainly those who do blog regularly -  Fred Wilson, Brad Feld etc. have an economic interest in doing so and put a lot of time/energy into their blogs.

It became clear to me about the time that Aggregate Knowledge really started to take off that I would have to make a choice about how much to blog. Spend time blogging or spend time making the business grow. In my case, the two aren't mutually compatible. Me blogging and participating in the conversation going on in the blogosphere isn't going to move my business forward that much. So I don't do it that often.

I sometimes think about the people who blog a lot the same way that I think about people who go to a lot of conferences. Don't you have a day job you should be doing?

Brave New Web 2.0 World

My friend Kevin did a session at SXSW on Breakup 2.0.

http://www.cnettv.com/9742-1_53-32479.html

Unfortunately CNet won't let me embed the video on my blog. :(

China is Fearless

This year Courtney and I headed out to China to visit my parents again. We go back every few years and I'm always struck by the amazing amount of progress and change that greets us. The pace of change in China is like that on the Internet but in the physical world.

3 years ago when we were in Guangzhou (located about 3h from Hong Kong) we flew through what would be considered a pretty typical airport in the US, this year as we flew a brand new

Since the last time we were in Hong Kong in December of 2006 a brand new Ferry terminal awaited us and the race was on between Shanghai and Hong Kong for the world's tallest building (neither city will disclose the height of their buildings yet).

Change and modernity are viewed as not only positive but as necessities. Growth is taken for granted in a way that it is not in other parts of the worlds including North America. As much as the US might have been the home of capitalism once that mantle has been taken up by China now.

China is willing to prioritize for growth and doesn't think twice about it. For example, think about all the rules and regulations that govern what can be built here in the US. How often does the skyline change in our cities? I have lived in San Francisco for 7 years now and with few exceptions there has not been a large amount of change in our skyline. Yes, there's a new building going up by the Bay Bridge but it is the exception that proves the rule - it's one building in 7 years and it has taken a long time to get done. There are an incredible amount of rules designed to keep things as they are instead of allowing htem to grow.

China also has access to a labor pool that is rapidly becoming more skilled and higher value. In contrast, the US is wracked with indecision on what to do about immigration. In my mind the answer is clear. We must enable a young dynamic workforce in the US. Many from Central & South America who are looking for better lives need to be allowed freer access to our markets to help build the economy. Failing to do this means that China will have an important lever for growth that we lack. It won't catch the US now but in another 20 years it could be a devastating difference.

This amount of change is driven by a fearlessness in the culture. In many ways it is easy for them to be courageous because they have little to lose and the alternative to continued growth is terrible and immediately obvious. We in North America have much to lose and so it is easy try and maintain the status quo. You can see these attempts in everything from our zoning policies, to what we subsidise, to who we do and don't let in the country.

Ironically it is these attempts to maintain it that will slowly but surely erode it.

Ok maybe it is a bubble...

I thought that this company reviewed by Duncan Riley on TechCrunch was a joke but evidently not...

From the Digital Telepathy website:

"The Design Products take you from concept to fully-functional online business in only 90 days."

"You bring the great idea and give us a few weeks of your time. We'll bring our veteran team and our proven process. We'll then head into the kitchen and mix up a "Biz In A Box" that'll be ripe for the market."

The most bizarre thing is that Duncan (who should know better) makes the following statement:

Prices range from $15,000 through to $250,000, although each project is calculated on a per startup basis. It may sound like a lot of money, but it’s a lot of service for your buck, and it is competitive.

It seems a little ludicrous to me that some company is going to go figure out your business model for you if only you have a great idea. If it only was so easy to have a "Biz in a Box"!

Why $240 Million could have been a deal for Microsoft

Ok so everyone knows about the news that Microsoft paid $240 M for 1.6%  of Facebook. Terrence Russel at Wired thinks it was a good deal and I do too but for different reasons.
 

I did a little checking on Google Finance and there's something interesting that happens. Once the Microsoft / Facebook rumor gets out and is judged to be serious the stock jumps 1%. It then jumps another 1% in after hours trading when the announcement is confirmed. On a market cap of about $290B that means that MSFT was valued at an additional $ 6B after the deal.

I'll admit I'm not an expert on the market but it sounds like turning $240M into $6B is a good move.

Radar Networks coming out of Stealth

My friend Nova Spivak's company Radar Networks is coming out of stealth mode! They are launching Twine.com which is a consumer facing semantic web site.

I'm very curious to actually get to play with it as I've heard about it for a while now but he's been keeping the details very secret.

Some more details by Martin LaMonica, Nick Carr, Download Squad, Wired and ZDnet.