What of the open source movement?
Netflix’s competition to rewrite their movie recommendation engine?

give me answers this qus….

In that sense, crowdsourcing sure does create new things, since a crowdsourced economy creates more new things than a non-crowdsourced economy.

No, no, I am not dismissing their value. Like I said above, crowd wisdom (of which PageRank is one example) does yield very safe (translation: acceptable, appropriate, relevant) predictions. Safety is not bad, just like comfort food is not bad. So I am not dismissing that value.

It just does not yield any predictions that are all that surprising or insightful.

The reason Google took off is that it succeeded in weeding out the spam to find the one, true home page for a person or company. Google’s main contribution to the information retrieval world was quick, effective navigational (i.e. “known item”) search. In other words, Google succeeds when the answer to your question is a “safe” prediction on which everyone can agree. So Google works for finding people’s home pages. Or company websites. Or product websites.

But the utility of PageRank (and other related crowdsourced methods) tails off extremely quickly. It very quickly gets the obvious correct navigational pages. But when it comes to something that requires more insight, it breaks down. And fast!

In other words, PageRank is good for recommending “The Who” for people that also like “The Rolling Stones”. But when it comes to finding lesser known bands, PageRank falls apart rather quickly.

So try a little experiment with me. Go into your Google “advanced settings” and set it to return 20 results per page, rather than the regular 10. Then ask Google a question like “What was the cause of the invasion and war in Iraq” (or, translated to Google shorthand, “cause reasons war invasion iraq”. Something for which there are literally hundreds of thousands of relevant web pages. And then do a little experiment. Look at the 15th result. (or the 12th or the 18th, I don’t care). How good is that result? How relevant is that result? I’ll bet you a 100 to 1 that it’s mediocre at best.

Or, if you don’t like that query example, then extend our little experiment further. Set your advanced settings to return the top 20 hits, like before. But now, over the course of the next week, for every query you do, don’t immediately click the 1st or 2nd result. Take a few extra seconds of your time and go down to (again let’s say) the 15th result. And just look at it. Open it up in a new tab if you want. But look at it. Think about how relevant it either is or is not. Then look at how many pages Google returns (typically in the 100s of thousands if not millions). And ask yourself: In those millions of pages Google has in its index, do you really think that this result, ranked 15th, is better than those undisplayed millions? As yourself, has the PageRanked linked wisdom of crowds succeeded in finding not just the obvious 1st or 2nd relevant page, but the 15th? The 30th? The 400th? There are topics for which there are literally hundreds of really good pages. How well does Google do in helping you find those? How quickly does the quality of the search results tail off, after the 5th or 6th ranked SERP?

Don’t let me answer this for you. Answer it for yourself, by doing a little experiment over the next week and consistently looking at the 15th ranked result. Then, let’s meet back here in a week, and continue this discussion about the ability of crowdsourced methods to yield insight and discover knowledge.

I’m serious. I could very well be wrong about this. That’s why you have to do the experiment, for yourself. I’ve done it for myself, but I’m only one person. But my feeling is that if crowdsourced methods were really able to “work to create new things”, i.e. to find those “ah hah” results that truly yield new and interesting insights, then the Google result at rank=15, for a query for which there are literally hundreds of thousands of hits, should be an extremely high quality result. See for yourself how often it either is or isn’t.

So, want to meet back on this page on Monday, March 17th, and see how the experiment went?

…and just try telling me the crowd’s book reviews on Amazon won’t sway your judgement to buy one iota

Oh, sure, yes. Of course I use those, and those help. I like that Amazon has them.

But I have gotten much better recommendations from a solitary friend (an “expert”) that already has a particular book and knows me well, or that has already used a particular product. I get more insight on photography books from my friend who is a professional photographer than I do from Amazon. I get more insight on gardening books from my hippie college roommate who tends her own garden, than I do from Amazon. I get more insight on cooking books from my friend that runs a restaurant, than I do from Amazon. I get more insight on Java programming books from my friend that works at Sun, than I do from Amazon.

It is from those people with working, practicing knowledge of a particular subject that I get real insight. Not from crowd reviews.

You may be right in that more problems will arise in which the right mixture of conditions can be found, which make crowdsourced methods possible. But to agree with your Bill Joy “the smartest people work for someone else” quote: I would rather just go to those people and ask them directly for their recommendation, rather than rely on a watered-down, statistically averaged, crowdsourced filter.

I wouldn’t be so quick to dismiss the value of these knowledgebases. It’s what helped differentiate Google for a start.

I haven’t read the book, though. I’ll check it out.

…and just try telling me the crowd’s book reviews on Amazon won’t sway your judgement to buy one iota

“it will never make those key inferences”
If you can point to instances otherwise..

So, back to soliciting crowd input for something more “meaty” than opinions on products…

I think it’s hard to cite examples of “tough” problems being cracked by crowds – there has been limited opportunity to do so. Netflix is the only public example I have experience of. However, the lack of examples shouldn’t necessarily be viewed as an indictment on the skills of the amateur. It’s not always possible for corporations to present a problem in the way Netflix did where all the necessary ingredients for this innovation were in place i.e:
* the company feels commercially comfortable airing the problem (and potential solutions) in the first place
* the necessary incentives are in place to motivate outsiders
* the problem does not require participants to have significant resources (e.g. not “curing cancer”)
* there is a manageable quality filter for companies to assess the crowd feedback at scale.

It’s rare that all these pre-requisites are in place but I don’t doubt that given these conditions answers are to be found. I like the Bill Joy quote :‘No matter who you are, most of the smartest people work for someone else.’

Yes, I am familiar with the author. I haven’t read the book, though. I’ll check it out.

However, there is definitely something to be said for harnessing opinions of large crowds. I think the closest thing we currently have to “intelligent” systems are those informed by the collective opinions of many people. It’s that blend of algorithms *plus* many human viewpoints that can produce the smarts.

Yes, that’s true, but again: What smarts do such systems produce? What insights and wisdom do such systems offer?

When I look at the evidence, yes, I see that such systems are good at filtering spam. But beyond that, the sorts of things that I learn from such systems are things like “people who bought hot dogs also bought hot dog buns” and “people who like The Rolling Stones also like The Who” and “people who watched Star Wars Episode IV also watched Star Wars Episode V”.

In other words, “wisdom of crowds” predictions yield very safe, very obvious connections. How is that going to really help me? I already knew all that, about the hot dogs and The Who and The Empire Strikes Back. Are those bad recommendations? No, they are not bad. They are spot on, in fact. But they are so spot on, that anyone could have told you that. No need for a fancy crowd-aggregation mechanism and 400,000 servers to crunch the numbers.

Which is why, in my very first post on this thread, I said: “Crowdsourcing will never teach us something that we didn’t already know…it will never make those key inferences that are necessary to truly gain new insight or wisdom.“

If you can point to instances otherwise, I would be interested in hearing about them. But all too often, the glory of crowdsourcing comes in recommending hot dog buns to people who buy hot dogs. True, and relevant. But ultimately not very interesting and useful.

You may find “cult of the amateur” an interesting read that has this as a theme. I didn’t agree with much of what was said in the book but it is an antidote to all this web2.0 zealotry.

“I see this as a small, focused, unified group vs. crowd issue”

Of course the old maxim “too many cooks spoil the broth” can hold here. For some small focused tasks a large crowd is undesirable.

However, there is definitely something to be said for harnessing opinions of large crowds. I think the closest thing we currently have to “intelligent” systems are those informed by the collective opinions of many people. It’s that blend of algorithms *plus* many human viewpoints that can produce the smarts.

Of course the viewpoints typically need filtering:
* In Amazon reviews or PageRank, attempts to game the system must be filtered out (by the community and/or algorithms).
*In Netflix a very formal ranking measure (RMSE) produces a leaderboard.
*In open-source projects contributors have to earn their spurs and are only granted rights to change code after proving their worth

If you can implement an appropriate quality filter and put in place the necessary incentives then there is wisdom to be had in the crowds.

I do have sympathy for this kind of frustration. I have felt it as well.

One example I bump into regularly is a distrust of open-source based on the unfounded belief that “the crowds” produce inferior software.

For what it’s worth, I am not one of those people that thinks crowds produce inferior software. I think crowds can produce software that is just as good as “professional” or closed software.

Where I remain unconvinced is whether crowds can create better software than a small, closed group of “experts”. I think there is a feeling amongst folks (maybe) like yourself that the crowd can not only do just as good, but can also do better. I don’t believe it.

I say this with no agenda against the crowd, or for the corporation. But I just do not see a lot of empirical evidence for revelatory, large-leap-forward crowd-developed software. Or movie recommendation. Or protocol development. Or user interface design.

Another example is the reluctance to share absolutely anything outside of the company for fear of losing “competitive advantage”.

I agree, that’s pretty lame, as well. In fact, I have seen certain companies simultaneously call for more openness in one arena while at the same time remain staunchly conservative and closed in other arenas.

Another annoyance is the arrogance of the “not invented here” syndrome.

I agree with you here, too.

For me, GWT and Netflix are positive examples of companies recognising the potential value of opening up to outsiders and having the courage and skill to harness that effectively.

Fair enough. Again, I wasn’t reacting to those particular concerns of yours, so much as I was reacting to the (what I believe to be largely unjustified) almost religious, dogmatic belief among many Web 2.0 entrepreneurs nowadays that the crowd can produce software/ ideas/ architectures/ designs/ user experiences/ etc. that are better than what a small, focused team can produce. Not as good as. But better. I just don’t see the evidence for that.

For what it’s worth, I don’t see it as a corporation vs. crowd issue. I see this as a small, focused, unified group vs. crowd issue. That is to say, the small, focused unified group does not necessarily have to be within a large corporation. It could exist in academia. Or it could exist in a proverbial Silicon Valley garage. But the insights and wisdom and intuitive, forward-landing leaps that happen get made, I believe, almost always by small, focused teams. Not by the crowd.

The crowd has the ability to maintain software, to debug, and to copy features that it sees elsewhere. And by so doing it can produce software that is just as good as other software.

But I have to agree with Jay Rosen, above. It does not work to create new things.

Absolutely – ditto for my comments.

I now think any debate on what is or isn’t “crowdsourcing” is fruitless. It seems there can be many possible interpretations.

I suppose most of my comments come from the personal frustration I have with typical short-sighted corporate mentality and their reluctance to “open up”. One example I bump into regularly is a distrust of open-source based on the unfounded belief that “the crowds” produce inferior software. Another example is the reluctance to share absolutely anything outside of the company for fear of losing “competitive advantage”. Another annoyance is the arrogance of the “not invented here” syndrome.

For me, GWT and Netflix are positive examples of companies recognising the potential value of opening up to outsiders and having the courage and skill to harness that effectively.

But again, I have to disagree when you say:

The windows analogy is not accurate or useful. Microsoft sell[s] Windows. It is their main source of profit and core to their business. You don’t freely get the source code and likely never will. They want to popularise their platform because it means revenue.

You say that just because MS sells Windows, that somehow opening the DirectX API to gamers is not crowdsourcing? That in order to be crowdsourcing, there needs to be open source so that people can innovate at all levels of the stack? I simply do not buy that. Why does that have to be true? All you need is an open API, not open source behind that API. Above, you say that crowdsourcing is:

“taking a task traditionally performed by an employee or contractor, and outsourcing it to an undefined, generally large group of people, in the form of an open call”

An employee of Microsoft, writing a game for Microsoft Game Studios, is not going to have to crawl through the source code to Windows in order to do their job. All they are going to have to do is know what the APIs are, and how to call them.

So if crowdsourcing is simply letting the crowd do what a MS employee might have done, and the MS employee didn’t need the Windows source, why should the crowd need it?

And what does selling Windows have to do with anything? At the end of the day, Netflix also sells a service. Netflix gets a bunch of people to work on the problem for it, but at the end of the day in order to take general advantage of the solution (“test” data rather than “training” data) you need to become a Netflix member, i.e. pay money for the Netflix service.

So does that mean Netflix is not crowdsourcing, because Netflix sells the service?

Microsoft sell Windows. It is their main source of profit and core to their business. You don’t freely get the source code and likely never will. They want to popularise their platform because it means revenue.

Google make no money from GWT. It is not core to their business. You do get the source to the whole thing. Why? In their words : “GWT took off much faster than we expected, and it quickly became clear that the most sensible way to advance GWT would be to open [source] it sooner rather than later”. This is from their FAQ.

The whole of GWT is open (not just the “surface level” as in the Windows API) and there are people innovating at all levels in the software stack not just the “widget” level.

It *is* different.
This is about a business taking the decision to turn over thinking to outsiders in the hope they will advance that thinking.

We are perhaps drawn back to the distinction between “platform” and “widgets” and “how many make a crowd?”. The more complex “platform” elements which you single out as the preferred subject of this discussion will invariably require greater skills which are in shorter supply and so there may not be such a huge “crowd” involved there. The size is not important. The point is Google found external resource that were both motivated and capable of advancing their thinking.

Interesting discussion!