When there's an 800 pound gorilla in your space, trying to steal bananas isn't exactly the smartest approach.  You figure out what he's not eating, and you start nibbling.  Before you know it, you're eating just as much as he is and wouldn't you know it, one bad banana crop and he's toast. 

So when I hear that someone wants to build a better search engine than Google, while I don't think it's impossible, I question along what lines they're trying to do it.  Can you really do it by indexing more pages than Google?  I find that hard to believe, because an infrastructure arms race with Google seems like a bad idea--no matter how efficiently you think you can manage your crawling costs.

Smarter algorithms?  Maybe, but isn't algorithm quality a function of sheer brain power of your search team?  Again, this isn't where I want to take Google on head to head.

No, where I think you can beat Google, or at least make some headway on them, is with people.

At the end of the day, computer interpretation of human behavior and desires is what drives Google.  You could attempt building a bigger or faster computer, but no one computer is really going to be able to interpret people better than, well, people.

That's why del.icio.us had so much potential.  Google could never figure out what was funny or interesting, but del.icio.us could. 

To me, it's also the reason why Firefox gained so much ground against Internet Explorer.  It wasn't that smarter people work on Firefox--it's that more people worked on only the things they cared about, solving problems for themselves.  The best ideas floated to the top and became part of the codebase.  Things got addressed that weren't a priority for the IE team, but that more engaged users had keen insight into the value of.  The more you directly involve people--at scale, which isn't easy--into the process, the better your product is, because your product is made for people.

So, right now, Cuil and a number of other startups have teams of a handful of people who are supposed to know better than all the Google people what users want out of their search and how to search better.  Why not, instead, open up the process to something more open source--more Firefox-like?

Here's what a more collaborative approach to building a better search engine might look like:

Backend:

Outsource just the basic crawl to Amazon, because they've probably got the best shot at competing cost effectively, but enable outsiders a chance to add elements to the crawl.  So, if you have a way of categorizing pages, like Cuil says they do, add that ability to the Amazon powered crawl, and your special taxonomy and tags will be available for anyone to access, work on and improve. 

Let others use your infrastructure to target specific pages with a different type of crawl and contribute to the results.  In other words, let Indeed and others run their crawlers on your infrastructure, so that the barrier to create new attempts at search isn't set artificially high.  This will make a lot more sense when we talk about Plugins.

Plugins

There are lots of different types of search that Google just doesn't do well, like jobs and events.  This has given rise to some opportunities in the vertical search market   Let's take that Indeed example.  Right now, Indeed searches jobs much better than Google does, so why not enable Indeed, Simply Hired or anyone else crawling jobs to outline what represent job keywords and searches, and automatically provide results for them.  You could even randomly rotate which job search engine plugin powers your job search and let them duke it out for highest clickthrough rates, or allow the user to set a default. 

Basically, a "plugin" would be a hosted version of the third party crawler that gets sent queries based on their structure, keywords, etc., and gets to send back all the results they can, as well as gets the opportunity to advertise against them.   So, in our "open source" search engine, when I type in "marketing jobs, new york, NY", instead of getting a page of links to search engines for their marketing jobs queries--i.e. an "extra click"--I'd actually get jobs as my results, and Indeed powered job ads.

The same could go for movies.  How many times do you type a movie name into Google, knowing full well that IMDB is going to be the first result?  Why not allow IMDB to be the movie plugin?  They could directly provide structured results for all the actor and movie queries and be allowed to advertise against them.  This way, you eliminate the Google middle man when all you were really trying to do was reach IMDB in the first place.  All you'd need are some standardized display templates for results, which could also allow some interface flexibility for different types of queries, like videos or location searches on local maps.

People could build other types of plugins, like one that would automatically display RSS results when blogs came up high in your ranking.  I get Google results for "Charlie O'Donnell's blog"...  let Newsgator build that plugin and power it with all of the clickthrough data on what my interesting most recent posts were.

The system of sending queries to the right search tool would be a kind of AdWords platform, but a level up the chain.  Instead of a marketplace for advertising next to one kind of search result, you'd have a marketplace of search results, each coming with their own ads in tow (or using a default ad platform that anyone could use.)  You could attempt to "buy" certain keywords to put your search results next to them, but you'd have to get good clickthrough performance to keep appearing.

Personalization

You'd definitely allow users to add their own scripts and plugins, as well as have them contribute other types of data.  Let me pump in my blog, my del.icio.us tags, twitter feed, etc. in an effort to teach the search engine all about what I like.  Let me remove results, follow my clicks... learn about me (and my friends) as I go along.

The company that should really get into this is Yahoo!  They couldn't out-Google Google on search or monetization, so they should just crack open the whole thing and let the community and other companies have a shot at it.  They could be the default ad network for searches that weren't powered by plugins...   and they could strike deals with plugin providers to take a smaller cut than Google would have had on ad clickthoughs. 

If not, I still think it would make for a pretty viable community project.  Hell, maybe Mozilla should be the one to work on it, or are they getting paid too much by Google for Firefox default search to rock the boat there?

Links I've recently tagged on del.icio.us:

Links I've recently tagged on del.icio.us:

Links I've recently tagged on del.icio.us:

"So we are thrilled to be an investor in a company that has been organized since its inception around the key insight that we believe will drive the next several years of innovation on the web – the need to solve real problems in the real world for real people."   - Brad Burnham on USV's Investment in Meetup

If only there were other companies solving real problems for real people.

Have you ever written something you really thought had a chance to go viral.  Maybe it wasn't even linkbait--it was just something you thought was really original and thoughtprovoking, and if only the right handful of people, or a critical mass of your friends caught onto it, it would take off.

So what do you do?

You bite the bullet and you start asking for people to Digg it.  Ugh.  There's nothing worse than being a social media panhandler.  It makes you feel stripped of your authenticity.  You might as well be pushing Rolexes.

So, what if, instead of blowing so much social capital on Diggbegging, you could easly throw a few bucks towards it to get it in front of the right bloggers.

Feedburner had something a little bit like that--where you could figure out what the CPMs were on your blog and advertise your feed in the feeds of other people in order to try to get more users. 

For businesses, easy tools like Clickable are really lowering the overhead it takes to have your own SEM strategy.  What individual contributors need is a kind of mini-Clickable right at the point of creation.  How great would it be if, right next to the publish button, there was another button that said, "Do you want to invest $10 in driving relevant traffic to this post?"  The money could be used to not only do search placement, but sponsored placement as "Suggested sponsored links" in front of bloggers talking about the same things.  They wouldn't get paid to reblog, so it wouldn't be like PayPerPost--you'd just be paying to get in front of people who are likely to reblog the idea.  

Another aspect of this is a better way to actually get your friends to reblog, Digg, tweet, etc. on your behalf.  What if I could sent one link around that gives people a menu with the minimum number of clicks necessary to spread it.  They could have their blog id/password, twitter password, digg, etc. already saved, so all it would take are some checkmarks and some additional optional descriptive copy in a few places and press "spread".  There are lots of times I would reblog something relevant that my friends send me, but I just don't have a lot of time to crack open the blog editor, do the appropriate cutting and pasting, etc.  This would come in especially handy when people send me jobs to repost.  I'd repost most of the jobs that people send my way, but it's kind of a pain in the ass.  If I could create a trusted list of friends that are allowed to repost their job ads to my blog, and maybe even aggregate them, I'd pre-approve people to do it.

Because who really wants to be a social media panhandler?

more cat pictures
more cat pictures

I seem to be the only human being in the world, but I was kind of disappointed by the Dark Knight. It was still a good movie, but definitely wasn't as good as Batman Begins. I don't know if it tried to do too much, but midway through the movie, I kept thinking that there was just too much plot going on here. I thought that truck flip scene was going to be the "final standoff" but there was still like an hour left in the movie.

It wasn't so much that there weren't good parts, but there were too many things that bothered me for this to get anything more than a 7, whereas the first one was close to a 10.

Here's what I didn't like:

1) Who is the Joker? In the original movie, we get the Joker backstory. The Joker was Jack Napier--an ambitious thug who falls into a vat of chemicals and gets a backalley hack to try to fix his face--only to leave that ridiculous smile on him. This time, we get nothing but the Joker keeps asking people "Wanna know how I got this scar?" and keeps coming up with new versions of the story. That's completely annoying because these new movie versions are all about the backstory. I wanted to know.

2) The whole Rachel Dawes character is a failure. What exactly did she do to warrant the love of two great men?  Why does Bruce Wayne love her?  Because they played together as little kids?  She's kind of vapid in this one--barely even caring that Harvey Dent nearly gets blown away in the courtroom. (The gun just happened to misfire right? Am I the only one that thought everyone should have reacted a little more to that?) I didn't care at all when she died. Not only is she no MJ from Spiderman, who I think we all fell in love with, but Maggie Gyllenhaal looks distractingly like Kirsten Dunst.

3) I really hate the Batman voice. Why is Christian Bale doing his worst Al Pacino impression under the cowl? It didn't bother me so much in the first one, because Batman doesn't really have a ton of dialogue, but in this one, it just gets silly.

4) The Batcycle is Bat Feature Creep. I was waiting for the back wheel to get blown off so it could get converted to the Bat Segway.

5) Too much obvious foreshadowing. How many "face" references to Dent were there? Face of Gotham? Hmm... I wonder if he becomes Two Face at the end? Lucky for him that Two Face was already his nickname!

6) Phone sonar: We've seen that effect before--in DareDevil, when it rains.

7) The Bat copycats expose us to the silliness of the idea of a guy running around dressed as a bat. What I thought was amazing in the first one was how believable the origin story of a guy in a bat costume was... Like I could actually see it happening and not being silly. When the copycat asks Batman what the difference between him and Batman was "I'm not wearing goalie pads" can't be the answer, otherwise Batman really is just a guy with a boatload of cash and military equipment.

8) Since when do all of the killings in a superhero movie have to be accounted for?  I hate the idea that Batman has to take the blame for Dent's murders.  Did he really kill 5 or 6 people?  I wasn't counting?  Are they counting Morone's limo driver, too?  Who cares who killed a mob limo driver?  Can't they sweep that under the rug?  Leave it as a cold case?   

9) His death was tragedy--no doubt. But, just in terms of his performance, I honestly thought it was kind of understated. When I think psychopath in clown makeup, I'm thinking REALLT creepy. Robin Williams was once mentioned as a potential Riddler before Jim Carrey got it, and his performances in 24 Hour Photo and Insomnia were beyond creepy--they were downright disturbing. I just didn't feel like the Joker was really that unpredictably nuts in this one. I did get my hopes up in the beginning, though. Between the "What busdriver?" line and the amazing disappearing pen, I thought we were in for a real sicko... in a funny kind of sick way. Didn't get him again until he walks out of the hospital in the nurse outfit...not much in between.

Links I've recently tagged on del.icio.us: