By Jason Calacanis

Every day a couple of dozen folks email me asking me to tell them what I think of their products.

When I have the time, or interest, to respond I generally ask myself a battery of questions:

a. Is the logo iconic?
b. Could you tell me the domain name over the phone once and have me type it in correctly?
c. Is it clear within 10 seconds what the purpose of this product or service is? Does it matter if it is clear?
d. Is the design world-class?

Those are tactical questions that let me know the person has skills. Everyone can pass those tests with a couple of months of hard work.

My two brutal questions, which you can't simply pass with a couple of months of hard work, are how recommendable and unforgettable your product is.

Here is how I ask them:

1. Would I recommend this product?
2. Will I remember this product next month and next year?

If you run an amazing product like Yammer, Zappos, Airbnb or Dropbox through the tactical questions, it quickly becomes clear that these firms have people with skillz.

a. Iconic logos? Yes, yes, yes and yes.
b. Domain name spelling test? Pass, pass, pass and pass.
c. Is the product offering clear? Yes: Collaborate with your coworkers, big photos of shoes and find a place to stay are all super clear. Dropbox takes a little risk by not stating clearly "share files with your co-workers and family." The video is awesome, but that's a slightly risky choice for a homepage.
d. Is the design world-class? Yes, yes, yes and yes. Are craigslist and eBay world class? Hell no, but those startups were both launched in 1995. They existed before world-class design was the entry fee.

The rubber really meets the road with the two killer questions.

1. Would I tell my friends about Yammer, Zappos, Airbnb or Dropbox?
-=-=-=-
Would I? I recommend these products dozens of times each month. Any time someone tells me they have a communication problem in their company I say, "Get Yammer." Any time someone brings up customer service I blurt out "Zappos!"

People are lamenting the cost of hotels? "Did you try Airbnb?" Need to share files right now? "Download Dropbox."

I've sent tens of thousands of folks to these products. They are 10s on the "would I tell my friends about it?" test.

That test is, of course, based on the Net Promoter Score. We're going to get into that deeper in a minute.

2. Will I remember this product next month and next year?
-=-=-=-
The second question is my own invention, and I call it my "unforgettable" test. Is this product so extraordinary that it will haunt me? Is it so well done that I'm not going to forget it?

When David Sacks showed me Yammer during rehearsals for my TechCrunch50 show, it burned itself into my brain. An open platform for everyone in a company to communicate. Wow. Twitter or Facebook for the enterprise. That's scary, awesome and empowering, and as a result it is unforgettable.

Yammer changed corporate culture so meaningfully that dozens of folks copied it.

Zappos' customer service is so unforgettable that people can't shut up about a company that does something as simple as put shoes on your feet. Think about that for a second. They kick ass so royally in customer service that you *want* to talk about buying shoes.

For folks who use Airbnb -- and I never have and probably never will* -- it's a religion. Their life experience has now been broad-ended because they can travel anywhere at rock-bottom prices and make friends with locals. For folks in need of quick, off-the-books cash, it's saving their lives. It's so compelling that before I go to a city, I find myself looking at search results I'm probably never going to use.

You know you're notable when people talk about your product even though it's not designed for them, and in fact browse it just to keep up with it.

Really, it's genius.

[ Note: I'm not against renting vacation homes. In fact, I'm renting a kick-ass place in Noe Valley for the rest of the month, and a place in Tahoe for a week this winter. It's just that I can't find those kind of three or four bedroom, family-friendly places on Airbnb. They're all on VRBO.com still it seems. I will keep checking, however.]

The promise and name of Dropbox are infinitely memorable. Free storage, works on every platform, integrated into existing file system. It's so good you ask "Why didn't this exist?"

Do you need to pass these tests to succeed?
-=-=-=-=-
Most products and services don't pass these tests. Try and think of an airline, cable company or wireless provider that's memorable or promotable. Okay, stop laughing and now look at how many billions of dollars these folks make and ask yourself why they don't have to pass these tests to be successful.

The answer is typically they have a monopolistic, highly defensible and at-scale business. They don't have to hustle, they don't need to be amazing, and since they have scale, they can actually squeeze additional profits out of their products by cutting down on the delightfulness.

However, if you are a startup, you don't have this luxury. Our (U.S.) government does not give out monopolies and scale anymore. Even space, defense and health care seem to be becoming more of a meritocracy (e.g., NASA and the USPS are losing their monopolies).

When I passed these tests and when I failed
-=-=-=-=-
In all of the big success and failure I've had in my life, I've passed these tests. Silicon Alley Reporter was memorable and people recommended it like crazy. A number of our blogs at Weblogs Inc. were notable enough for folks to remember and recommend: Autoblog, Joystiq, Cinematical, Hackaday and, of course, Engadget.

The LAUNCH Festival (and my previous creation TechCrunch40/50) created a lot of buzz.

Mahalo 1.0? Well, while I got the product to good, I'm self-aware enough to know that I didn't exactly get it to "tell your friends" status. I did strike a nerve with "human-powered search," but unforgettable + not recommendable = "nice try."

Now that Mahalo has pivoted to education, in the form of apps and videos, we've been having our notable and recommendable moments.

That's been really, really great for me and the team.

Our educational videos around guitar have done over 25M views on YouTube to date. Our "Learn Guitar" app has consistently ranked in the top 50 music apps since we launched it three or four months ago.

The Net Promoter Score really is the best way to get there in my mind.

These days I host meetings with my team where everyone rates a particular video on our semi-customized Net Promoter Score scale:

1 - 6: Bad, and so bad you might tell your friends NOT to get the app.
7 or 8: OK and good. I don't feel ripped off by this app but I'm not so blown away I pull out my phone and show it to someone.
9 or 10: Excellent and otherwordly. I'm so enthusiastic about this app I can't stop telling my friends about it.

How do you get to the 9-10 level?

1. Hiring and firing.
2. Iterating.

I know, it's a hard thing to say, but if someone consistently makes an okay product you need to fire them right now. If you make 6 and 7 level content, you suck and you're not doing the world any good.

If you're producing 7 or 8 level content, well, you need to work 30% harder and get yourself to an 8 or a 9. Folks who consistently produce 8 and 9 level content will, inevitably, have a 10 moment. Folks who consistently produce 7 and 8 level content will never hit a 10.

When your company is having honest discussions about doing good work, you're about to make great product. In fact that's where Mahalo and ThisWeekIn.com are right now -- and it's awesome. Folks can taste how close they are to the 9 or 10.

Being EXCELLENT at consistently producing GOOD product is OKAY.
Being EXCELLENT at producing EXCELLENT product is EXCELLENT.

Google is excellent at producing good product, but it is not excellent at producing excellent like Apple. Which proves that you can have a big business by being excellent at good, but to be loved you have to be excellent at excellent.

Does that make sense?

Bottom line: not everyone is going to create a 9 or a 10 every time they build something, E.S.J. (except Steve Jobs). But if you're not in the 8 or better range consistently, you don't have a place in startup land. Sorry.

To be honest, this is why I haven't written a book yet.

I think if I wrote a book three years ago it would have been a 6 or 7. If I wrote the book last year it might have been a 7 or an 8. Every month my friends send me their 6, 7 and 8 books and I read the first chapter, put it down and say, "They should have waited."

I'm waiting for next year or the year after, when I've got enough in me to write the 9 or 10 book.

Writing, like product design, is an iterative process. The more you do it, the better you get. Writing these emails to you and watching how you respond has made me the writer I am today -- which is to say one that doesn't suck, but one that is not going to write a 10 book yet.

Books and movies need to be 9s and 10s or they really should not be produced. It's too much commitment for the creator and audience. Of course, Hollywood's monopoly on films lets them make 10 returns on 7 or 8 films. Mission Impossible 4 is a solid 8 and Avatar was a 7 -- but they made absurd returns due to the rigged machine Hollywood has built.

What I've learned from self-analysis is that I know I can get you guys 8, 9 and sometimes 10 emails. How do I know this? Because I watch how often you forward, tweet, like and reply to each one.

That's why you guys typically get my emails in bunches. If 50 or 100 of you write back "good one" or "best one yet," I get motivated and I write more.

1,200+ of you tweeted the Cult of Amazon Prime story.

I hit the 10 mark on that bad boy -- and frankly I thought it was an 8.5.

If my email can do that regularly, that's when I start writing the book. If I can put together three 10s in a row, yeah, it's time to write a book. I'm not there.

Being self-aware enough to rate yourself on a brutal scale of 1 to 10, and then figure out what it will take to move slowly up the leaderboard, is critical to being an entrepreneur.

Most folks in the world can't handle this level of scrutiny.

Bottom line: If you can't get past a 7 or 8, you should really quit the samurai/startup business. The world doesn't need you to make products. Stop. Quit.

And remember, if you are not a samurai you are a rice picker.

Nothing wrong with picking rice, someone's got to feed the samurai I guess.

best, Jason

LAUNCH Blog