As an entrepreneur, I have brainstormed for startup ideas so many times, I’ve lost count. If you’re like me, you’ve probably done a lot of this too. Over the years, I started noticing that what I really wanted to find was an “aha” experience. The exercise became a bit like solving a puzzle.

In the 1970s and 1980s, management consultants used a puzzle to foster creative thinking with clients.

The idea is to think of a way to connect all of the dots using four straight lines or fewer, without lifting the pen and without tracing the same line more than once. Before, scrolling down, see if you can do it.

Here is the answer. There are several others but they are all rotations of this shape.

Once you get the “aha” experience of realizing that you can extend a line beyond the frame of the dots, the problem becomes fairly “obvious.”  This puzzle became the basis for the phrase “out of the box” thinking.

Startup ideas are often like this. They often involve an “aha” moment when the unobvious becomes an obvious solution.

We often want to get a similar feeling of novelty and creative problem solving when we are looking for that perfect startup idea.

Paul Graham put it this way in his amazing article How to Get Startup Ideas:

…if you want to start a startup, you’re probably going to have to think of something fairly novel. A startup has to make something it can deliver to a large market, and ideas of that type are so valuable that all the obvious ones are already taken.

 

That space of ideas has been so thoroughly picked over that a startup generally has to work on something everyone else has overlooked.

Coming up with a startup idea that no one else has noticed is daunting. It’s like sifting through a desert of used or bad ideas to suddenly stumble upon an overlooked startup gem hiding underneath all the other ideas. Because of its novelty, this overlooked idea will also have a built-in benefit: zero competition.

According to Graham, a novel idea with little to no competition is a key component to startup success. But don’t start pulling your hair out just yet trying to reinvent the wheel with something no one’s ever thought of before.

Lots of other gurus feel that industry expertise, execution, competitive advantages, etc., are far more important than novelty. I’m here to challenge the idea of novelty even further.

In this post, I’ll help you hit the mark by finding high-potential startup ideas tailored to you.

To help guide you through this article, I’ve broken it into three main sections. Feel free to skip to the sections you think are most useful to you:

Part 1: Ideation–Creative Thinking

Part 2: Evaluation–What Makes a Great Startup Idea

Part 3: Pre-Inventive Frameworks for Startup Ideas

Feel free to use the links above to skip around.

Part 1: Ideation–Creative Thinking

“Chance favors the prepared mind.” –Louis Pasteur

Writing an article on startup ideas without mentioning creativity would be like painting a rainbow with only variations of black and white; creativity is at the core of every startup idea. In this section, we highlight the aspects of creativity that are most important for coming up with great startup ideas.

Defining Creativity

While we don’t have time to get into the deep and esoteric meanings of creativity and novelty, a basic understanding of these two concepts is key to startup ideation, so we’ll touch on them briefly.

Most experts define creativity as producing something new and useful. At its core, creativity is basically a mixture of novelty and utility.

The Stages of Creativity

In 1926, Graham Wallas conducted pioneering research into creativity and came up with five stages of creativity. I have taken most of the following stages from the Wikipedia article on creativity.

  1. Preparation. Preparatory work on a problem that focuses the individual’s mind on the problem and explores the problem’s dimensions.
  2. Incubation. Where the problem is internalized into the unconscious mind and nothing appears externally to be happening.
  3. Intimation. The creative person gets a “feeling” that a solution is on its way.
  4. Illumination or insight. Where the creative idea bursts forth from its preconscious processing into conscious awareness.
  5. Verification. Where the idea is consciously verified, elaborated, and then applied.

In my own life as an entrepreneur, I agree completely with Wallas’ stages. But I would emphasize the importance of preparation.

In 1939, James Webb Young published his amazing book on creativity: A Technique for Producing Ideas. Years later, he published a postscript in the reprint of his book where he also underscored the importance of preparation:

From my own further experience in advertising, government, and public affairs I find no essential points which I would modify in the idea-producing process. There is one, however, on which I would put greater emphasis. This is as to the store of general materials in the idea-producer’s reservoir.

[…]

I am convinced, however, that you gather this vicarious experience best, not when you are boning up on it for an immediate purpose, but when you are pursuing it as an end in itself.

Given the importance of preparation and incubation in ideation, you can see why coming up with good ideas can take time. We’ll talk about that next.

Noticing versus “Thinking Up”

“Any sufficiently advanced technology is indistinguishable from magic.” –Arthur C. Clark

Startup experts like Paul Graham feel that directly trying to think up startup ideas is probably the least effective way to come up with good startup ideas. He puts it this way:

Finding startup ideas is a subtle business, and that’s why most people who try fail so miserably. It doesn’t work well simply to try to think of startup ideas. If you do that, you get bad ones that sound dangerously plausible. The best approach is more indirect: if you have the right sort of background, good startup ideas will seem obvious to you. But even then, not immediately. It takes time to come across situations where you notice something missing.

He goes on to criticize “artificial” startup idea processes:

The verb you want to be using with respect to startup ideas is not ‘think up’ but ‘notice.’ At YC we call ideas that grow naturally out of the founders’ own experiences ‘organic’ startup ideas. The most successful startups almost all begin this way.

He also highlights that many founders build things no one wants because they use these artificial processes:

Why do so many founders build things no one wants? Because they begin by trying to think of startup ideas. That m.o. is doubly dangerous: it doesn’t merely yield few good ideas; it yields bad ideas that sound plausible enough to fool you into working on them.

 

At YC we call these “made-up” or “sitcom” startup ideas. Imagine one of the characters on a TV show was starting a startup. The writers would have to invent something for it to do. But coming up with good startup ideas is hard. It’s not something you can do for the asking. So (unless they got amazingly lucky) the writers would come up with an idea that sounded plausible, but was actually bad.

I partially agree with Paul Graham. As mentioned above, I have personally experienced the importance of noticing, incubating, and coming to great ideas indirectly. But as a computer developer, I know many of these things are just human limitations.

If you built an AI to come up with startup ideas, it probably wouldn’t need an incubation period. Rather, it would figure out what your unconscious brain is doing during those incubation periods and run those processes explicitly.

This is why I started this section with the Arthur C. Clarke quote about magic. Startup ideation is only magic to us because we don’t understand it fully yet.

As we understand it more and more, I believe entrepreneurs (and probably computers–gulp) will increasingly be able to “think up” great ideas.

And, in fact, we are already starting to understand the creative process better. From research on the topic, I have found three basic frameworks to make the creative process more explicit: BVSR, Conceptual Blending, and Effective Surprise.

BVSR

We often think that brainstorming is the most basic form of ideation, but it’s actually part of a much larger concept called blind variation selective retention, or BVSR for short.

BVSR is a way to test and filter ideas to find the strongest one. This happens in nature, too, when living things go through the process of evolution. In evolution, nature selects the gene combinations that are most likely to reproduce. The ability to reproduce is competitive, leading to an increase of the fittest genes in surviving populations.

Because gene replication involves randomness, new possibilities are constantly being tested, and animals with new, successful gene combinations are more likely to reproduce.

Just like evolution, every brainstorming process involves two steps: ideation and evaluation. Ideation is just another word for “blind variation” and evaluation is just “selective retention.”

So, when you brainstorm, you perform a kind of rapid mental evolution, producing somewhat random ideas and then testing them against an evaluation model for success.

If you’re ever frustrated with the process of ideation, just remember that it’s a much quicker process than natural evolution!

Evolution is an interesting metaphor for brainstorming because it generally involves relatively small and incremental innovations and proceeds slowly. For example, while having three arms may provide an evolutionary advantage, nature hasn’t made that particular adjustment yet. I’m still waiting for the day when I can type and eat a sandwich at the same time…

BVSR has two parts: blind variation and selective retention. Blind variation is just another word for novelty and selective retention is just another word for usefulness. Evolution and BVSR processes lead to useful novelty.

Conceptual Blending

Conceptual blending is the idea of mixing two (or more) frameworks. Analogical reasoning, structure mapping (in AI), and bisociation are all examples.

Let’s look at Arthur Koestler’s theory of bisociation as an example of conceptual blending.

The basic idea of bisociation is that creativity comes from the sudden and unexpected merger of two (or more) frames of reference previously considered incompatible. Koestler describes it like this:

The pattern underlying [the creative act] is the perceiving of a situation or idea, L, in two self-consistent but habitually incompatible frames of reference, M1 and M2. The event L, in which the two intersect, is made to vibrate simultaneously on two different wavelengths, as it were. While this unusual situation lasts, L is not merely linked to one associative context, but bisociated with two.

 

I have coined the term ‘bisociation’ in order to make a distinction between the routine skills of thinking on a single ‘plane,’ as it were, and the creative act, which … always operates on more than one plane. The former can be called single-minded, the latter double-minded, transitory state of unstable equilibrium where the balance of both emotion and thought is disturbed.

The basic idea of bisociation is mixing conceptual frameworks and finding important similarities and differences between them.

One of my favorite classes at Harvard B-School was one where I learned about the Bass Diffusion Model. It’s a mathematical model for the virality of a product and network effects. It was based on transferring ideas from the study of epidemics to business products.

For example, who knew that modeling Facebook adoption is very similar to modeling the spread of infectious disease in a population? The epidemic model that Bass relies on in turn borrowed heavily from a chemical model called the Law of Mass Action.

Here’s another example. NPR had a piece last night about Aretha Franklin’s famous song “Respect.” It turns out she stole huge elements of it from Otis Redding’s song “Respect.” Aretha’s big innovation was that she re-interpreted Otis’ song from a woman’s perspective–giving it it’s timeless feminist appeal.

So, not surprisingly, mixing previously unrelated frameworks can lead to useful novelty as well.

Effective Surprise

“How extremely stupid not to have thought of that.”
–Thomas Huxley on being shown Darwin’s theory of evolution.

With regards to creativity, Jerome Bruner states, “An act that produces effective surprise [is] the hallmark of the creative enterprise.”

I have fallen in love with the idea that at its core, all creativity is just “effective surprise.”

When you look for it, you can find effective surprise in nearly every creative activity–jokes, art, science, and business–especially startups. If you can find a way to surprise people with a groundbreaking product or service, you should have a good chance at success.

In many ways, this is probably why young people create so many of the biggest startup successes. Young people are experiencing the world in a very different way than the old guard who is running it. So their business ideas are often surprising to the status quo. In fact, any minority group that frequently gets ignored by those currently running the world can often produce effective surprise.

To prove it, I did an analysis of the top 100 most valuable startups and found that college-age kids started 18% of them. They’ve essentially taken what comes naturally to them–seeing the world through new eyes–and translated this raw creativity into startup terms. In a way, you could say that creativity is finding a novel startup idea with high potential.

At first glance, the previous two major frameworks we discussed–brainstorming and conceptual blending–may seem to have nothing in common, but if you look closer you can see that the common link is effective surprise.

Instead of thinking about discovering novelty, I’ve found it useful to instead think about creating effective surprise.

Given the high failure rates of startups, every startup success is essentially a form of “effective surprise” in of itself (even if it isn’t novel). This quote from Tomas Tongusz really captures the essence of this idea:

When I started in venture, I took a naive view that with enough data, I could filter and find great startups to invest in. At Google, I learned how powerful data can be. But almost nine years in, I see how fallacious that perspective is. I’ve done the analysis.

 

You can find variables that correlate to success. You can increase or reduce the number of variables to examine. You can overfit models till you’re blue in the face. But predictive modelling will never capture exceptions. The models are based on the notion that they capture the typical outcome. What startup founder is typical? What startup journey is typical? All of them are outliers. …The odds are the business will fail.

We might even say that it is effectively surprising that any startup, given their outlier status, succeeds at all.

All of this has emphasized two major aspects of creativity: novelty and usefulness. We can now begin to categorize different startup ideas based on the extent to which they are are novel and / or useful.

The Novelty-Utility Matrix

It’s probably an oversimplification, but we can divide startup ideas into four basic categories:

In reality, nothing is as cut-and-dried as this chart. Some ideas hover near the boundaries and some ideas move from one category to another; they seem strange at first but become paradigm-shifting. Others seem incremental at first but become revolutionary.

We can also use the idea matrix to distinguish between two different types of startup ideation: those that focus more on utility and those that focus on novelty.

Utility-Driven versus Novelty-Driven Ideation

Sarah Kaplan’s new research on patent innovation shows that highly novel patents have the highest economic value. This view would seem to confirm the paramount importance of novelty.

But the research shows that these kinds of innovations are exceedingly rare. Fewer than 1% of all patents fall into this category.

This again seems to validate our hypothesis that focusing on novelty is like swinging for the fences. If you connect with the ball, you’ll probably hit a homer. But trying to swing with that much force decreases your chances of connecting with the ball at all.

Kaplan also highlights two kinds of brainstorming:

Basically, in research on innovation, there has been an idea that breakthrough innovations, those with the highest impact, are ones that are produced through … what we call “re-combination” or combination processes of distant and diverse knowledge… For practitioners, the important thing is to remember that not all creativity happens through combination processes. We have this idea out in the world that … bringing distant and diverse knowledge together is the way to get creative insights, and that’s certainly true. However, what we discovered is that there’s an equally important process of the deep dive, of deep knowledge in one domain.

She says this about deep-dive, expertise-driven brainstorming:

Brainstorming is not enough without deep knowledge development that you would need in a particular domain to understand what the issues are so that you can break away from existing ways of thinking.

Kaplan’s research invites us to consider two major modes of brainstorming. Brainstorming that focuses more on novelty (re-combination) and brainstorming that focuses more on utility (deep domain knowledge).

It turns out there are some big benefits to utility-driven brainstorming and some big risks associated with novelty-driven brainstorming. We can see this in our idea matrix.

If we limited our brainstorming only to “highly useful” ideas, we would only come up with two kinds of ideas:

  1. Paradigm-shifting / revolutionary ideas
  2. Incremental / evolutionary ideas

We might luck out and find a paradigm-shifting idea, but chances aren’t in our favor. Remember, these kinds of innovations are very rare. But we’d still have a great shot at finding some incremental / evolutionary ideas, and those can be very valuable!

If we limited our brainstorming to only ideas that are “highly novel,” we would come up with these two kinds of ideas:

  1. Paradigm-shifting / revolutionary ideas
  2. Strange ideas

Again, we might luck out and find a paradigm-shifting idea. It’s no coincidence that as Seneca said “no great genius has ever existed without some touch of madness.” But chances are, we’ll end up with a strange idea.

Some fields of human endeavor–like art–often seem to value novelty for novelty’s sake. We even have a phrase for this: “novelty act.” But strange startup ideas that aren’t particularly useful typically fail. In fact, strange startup ideas are basically the exact ones the PG Warning refers to–they are often bad ideas masquerading as good ones.

I think most fast-paced, forced ideation is novelty-driven ideation. This is what we typically think of when we think about “brainstorming.”

On the other hand, utility-driven ideation is quite different. Remember, BVSR teaches us that determining the usefulness of an idea is just “selective retention.” Knowing what to throw out and what to retain is often much harder than it seems.

It turns out, the key difference between traditional “brainstorming” and utility-driven ideation is expertise.

Innovative Thinking and Expertise

Let’s distinguish traditional brainstorming–which often overemphasizes novelty–from an approach I’ll call “innovative thinking,” which focuses more on usefulness.

Many people believe the best startup ideas spring naturally from expertise.

Arthur Koestler put it this way: “Minor, subjective [creative] processes do occur on all levels, and are the main vehicle of untutored learning. But objective novelty comes into being only when subjective originality operates on the highest level of the hierarchies of existing knowledge.”

I have often felt that “recognizing” great ideas is one of the central challenges of creativity. For example, I recently googled the term “best painting ever.” Here’s what I saw.

With apologies to my art historian wife, I was surprised to see a painting called Las Meninas by Velazquez at the top of the list.

At first blush, I couldn’t understand why it was in the company of other more obvious paintings like the Mona Lisa, Starry Night, The Scream, etc.

Reading on, I found a critic who called Las Meninas “one of the most overpowering paintings in western art history” while noting “It doesn’t seem so mysterious at first glance.”

At first, it just seemed like a very nice, traditional, albeit somewhat crowded, painting. On closer review, the painting reveals layers and layers of recursive questions about reality, illusion, art, and the artist.

It served as a nice reminder that sometimes a great idea is staring you in the face but you don’t have the expertise to recognize it.

Experts are far more likely to recognize great ideas in their industry. They are also more likely to know what has been tried before, what new ideas have the most potential, and how best to execute an idea.

Recent research has produced a lot of insight into how innovative people think. Clayton Christensen points out that the most innovative people do the following:

  • Associate seemingly unrelated questions, problems, ideas
  • Question conventional wisdom
  • Observe much more carefully the behavior of potential customers
  • Experiment
  • Network

Anybody can do this…but industry experts do it with applied knowledge and deeper insights, often leading to fantastic results.

Christensen points to Marc Benioff, the founder of Salesforce, as an example of questioning the status quo. Benioff didn’t think that software had to be installed, and from that break with convention, Salesforce was born. He also talks about how Scott Cook, the founder of Intuit, came up with Quickbooks by carefully observing his wife do their finances.

We can assume, then, that brainstorming is unnecessary for industry experts, as great startup ideas simply spring from solving problems they face every day. After all, when you talk to an industry expert, they often have a really great sense of the new “evolutionary” ideas affecting their industry. This also explains why industry experts rarely seek revolutionary ideas. With viable evolutionary ideas constantly staring them in the face, there’s no need for them to go the extra revolutionary mile.

But there are drawbacks to expertise as well. Sometimes it can limit ideation. In certain circumstances, experts disregard novel ideas that they see as too unconventional or that disagree with broadly-accepted truths of the day. (Just think about expert predictions for the 2016 U.S. presidential race.) Experts sometimes limit themselves by not thinking outside of the box of their own expertise.

Part 2: Evaluation–What Makes a Great Startup Idea

“As far as I can conjecture the art consists in habitually searching for the causes and meaning of everything which occurs. This implies sharp observation and requires as much knowledge as possible of the subjects investigated.” —Charles Darwin on how he came up with the theory of evolution.

In this section, we’ll challenge the idea of creativity and novelty and move toward a more encompassing theory for startup success. We’ll get to the source of ideation by first understanding what makes startups succeed so we can pave a clear path to success.

The Assumption that Novelty Equals Success

Let’s start with the hypothesis from our intro. If Paul Graham is right, novelty is one of the most important components of startup success.

Many people feel that novelty leads to success because novel ideas don’t have competition. But novelty has an important relationship with obviousness. Most obvious ideas were novel at one point and then became obvious once the idea spread.

Generally an innovator comes up with a groundbreaking/novel idea. In that moment, the idea moves from not obvious to obvious for the innovator (initial discovery). Over time, innovations become obvious to more people like subject matter experts and ultimately to all of society (knowledge diffusion).

Economists everywhere have been preaching that obvious ways to make money get competed away. This is the basis of the “Efficient Market Hypothesis.”

If a good startup idea is obvious, it should face competition. The competition will be more intense the bigger the growth/profit potential of the idea. Let’s call this the “Startup Efficient Market Hypothesis” or “Startup EMH” for short.

Startup EMH seems to explain a lot. It explains why so few venture capitalists generate good, risk-adjusted returns. VCs are placing bets in a startup market that has become quite efficient. It explains the anxiety founders feel around first-mover advantage, keeping an idea in stealth mode, etc. It even probably explains why many entrepreneurs fail many times before they succeed. If you only have an average chance at success with each attempt, it can take many attempts to win.

This would seem to imply that the best startup ideas should be really novel with big market potential.

To test this conventional wisdom, I wanted to discover what percentage of startups succeed with truly novel ideas.

Testing the Importance of Novelty

For this experiment, I analyzed the top 100 most valuable startup companies (aka, the top 100 “Unicorns”).

To estimate novelty, I gave each startup idea a novelty score of 1 (highest) to 3 (lowest) at the time of the startup’s founding:

  1. Globally Novel. Everybody would consider the idea novel including industry experts
  2. Novel beyond Industry. Everybody except for industry experts would consider the idea novel
  3. Obvious. Most business people would consider the idea somewhat obvious (i.e., not novel)

Here’s what I found:

Download my list and see if you agree with my categorizations or do your own analysis here.

The “globally novel” companies were some of the most highly valued startups on the list representing 5 of the top 20 companies:

  • Uber
  • Airbnb
  • Snapchat
  • WeWork
  • Pinterest

But only 16% of all companies in the set were globally novel.

Based on this, it would seem global novelty (what we think of as the pinnacle of innovation) accounts for a relatively small minority of Unicorn startup ideas.

And yet, novelty and creativity are still thought of as the integral keys to startup success. Why does this data prove otherwise for most cases? What’s more, if the data reveals that you can simply pick obvious ideas for a startup and still succeed, why isn’t everyone just doing that?

Novelty Entropy

Novelty wears off. We already discussed earlier how a “novel” idea eventually becomes an obvious one. Over time, as market participants notice a good idea, the idea will become less novel and more competitive. Essentially, novelty is entropic.

The fact that novelty wears off on a personal level is common sense. We get tired of doing the same thing over and over. Novel startup ideas also “wear off” but in a slightly different way. As startup ideas become less novel, more competition enters, making success harder.

The path of a startup with a novel idea will look something like this:

This seems consistent with startup EMH, but there is a crucial difference. The market doesn’t start out totally efficient (i.e., competitive), it moves toward efficiency.

This means that those who come up with novel ideas have a head start. This is the concept of a first-mover or early-mover advantage.

There are several important novelty advantages:

  • Ability to make money without competition for a period of time
  • Ability to secure venture capital (more and earlier)
  • Branding
  • Ability to learn earlier
  • Ability to get lock-in (network effects)
  • (If possible) an ability to secure a patent on an idea

Believe it or not, there are probably several novelty disadvantages too:

  • Pioneering an idea that others can copy with less effort / cost
  • Educating the market about an idea which will benefit all competitors
  • Inability to meet all of the demand

As a market moves toward efficiency it often passes through 3 stages:

  1. No competition and not enough supply
  2. Competition but still not enough supply
  3. Full competition and full supply

So ultimately–in the long run–a novel idea will become obvious, competition will enter, and novelty entropy sets in. This makes finding a source of competitive advantage necessary for winning in the long run.

Beyond Novelty: All the Ways a Startup Can Succeed

With this as a backdrop, we can now point to several ways a startup can succeed:

  1. Win through competitive advantage
  2. Win through market inefficiencies
  3. Win through little or no competition

Where does novelty fit into this? Novelty minimizes competition, so it fits into strategy 3. But that’s only one of three possible success strategies.

This explains why startups can (and frequently do) succeed with less-novel, or even totally obvious, ideas. We’ve now reached a point where it’s safe to say that novelty, while not entirely unuseful, is overrated.

Now that we’ve gotten that out of the way, we can look toward a more detailed, all-encompassing road map for finding great startup ideas. If you can find a startup idea that allows you to win in any of the three ways above, you should be in the money.

Let’s look at each category in a bit more detail.

Winning Through Competitive Advantage

Even the most novel startup ideas only give you a limited time advantage, as ideas don’t stay novel forever. True, patents can extend that advantage to over a decade, but not all ideas can be patented.

Hence, finding a competitive advantage is not only smart; sometimes it’s a prerequisite for success.

Competitive advantage can take on many forms, but we can simplify it down to three overarching types:

  1. Proprietary Resources. This is something that you have that others don’t have access to. In real estate, it could be a great location. For startups, it could be network effects, branding, etc.
  2. Differentiation and strategy. What segments are you going to focus on, how will your product or service be different, what is your strategy?
  3. Execution. If every competitor had the same resources, the same innovations, and the same differentiation and strategy, then the one that executes the best will be the reigning winner.

The last type of competitive advantage–execution–merits special emphasis. As we’ve mentioned before, most experts believe that execution is more important to a startup’s success than the startup idea itself.

Execution

When I worked as a venture capitalist, we used to say this:

An “A” startup idea with “B” execution is a “C”
A “B” startup idea with “A” execution is an “A”

Taken to its logical extreme, we might say that execution is the only thing that matters. A great management team can execute pretty well on even a mediocre idea. What’s more, great managements teams are good at finding better ideas even when they start with a poor idea.

There is a whole school of thought that encourages founders to see startups as experiments (like Lean Startup). If you are really good at running experiments and learning from them, it may be that the startup idea doesn’t matter at all, as long as your experimentation and execution lead to startup success.

Winning Through Market Inefficiencies

Is it possible to succeed with a non-novel idea in a competitive market even if you don’t have a competitive advantage? Yes. In fact, it happens all the time.

Your product may be inferior to your competitors, but if they can’t effectively fill all of the potential demand, then you can still succeed.

You may think that this is a risky strategy, but you’d be surprised to find that it’s one of the most common approaches Unicorn founders use as we will see in Part 3.

Imagine you came up with the idea for Uber. Once you saw the huge demand for your service, you would probably want to roll it out to every potential customer around the world.

But it doesn’t work that way. It takes a lot of time, effort, and money to offer your services in new markets. So you won’t be able immediately offer your services everywhere. This creates a space for potential competitors.

In fact, this is exactly what happened to Uber. Local startups in several geographies sprang up to meet the local demand, like Ola in India and Didi Chuxing in China. They took advantage of a market inefficiency–the time it would take for Uber to roll out its service to every potential buyer. So if you can take advantage of the market inefficiencies like this one, you could potentially see startup success.

Winning Through Little Or No Competition

“Competition is for losers.” –Peter Thiel

In this category, we aren’t talking about beating competition or competitive advantage, we’re talking about finding ways to sell a product where there is very little or no competition.

We tend to think that the only way to stumble into a magical no-competition market is by creating an idea that is so novel, it generates a market of its own (which we will discuss in the next section). But surprisingly, there are many ways even “obvious” ideas sometimes have no competition:

  • Niche ideas
  • Forgotten ideas
  • Unsexy ideas–ideas in businesses that are sleepy, old-school, or boring

For now, we’re going to leave this idea on the backburner until Part 3 where we’ll discuss ways in which having a competition vacuum can produce fantastic startup ideas.

Part 3: Pre-Inventive Frameworks for Startup Ideas

“Creativity loves constraints.” –Marissa Mayer

There is a prominent theory of creativity called Geneplore that posits the existence of structures that facilitate creative thinking–called pre-inventive structures. In this section, I’d like to apply this idea by exploring pre-inventive frameworks for developing startup ideas.

Since there are more frameworks than there is time for us to explore them all, I decided to narrow it down by looking into whether any successful startup founders used pre-inventive frameworks, and if so, which ones.

How Unicorn Founders Got Their Ideas

For my research, I pulled a list of the top 100 private company startups (“Unicorns”) and researched their founding stories to identify how they came up with their ideas.

I identified categories that best captured the startup idea approach each company used, and then I gave each category a point whenever I found a company that used that approach. Some companies actually used multiple approaches – in these cases, I gave a point to each approach used.

The percentages below represent the frequency of a given approach as a percent of the total number of observed approaches.

Download my list and see if you agree with my categorizations or do your own analysis here.

In my research, I was surprised to find a couple stand-out categories that I hadn’t previously considered:

  1. Grew out of existing business. 9% of founders got their idea by spinning an idea or business out of another existing business.
  2. Pivot. 8% of founders got their idea by pivoting from an original idea. Only one company actually cited using Lean Startup to find their idea but many companies used elements of the Lean Startup process.

I was further surprised to see the wide variety of approaches that resulted in startup success.

For instance, who knew that 11% of founders were basically copycats? It’s an unanticipated fact that goes against a lot of conventional wisdom about avoiding competition. But, about half of the 11% avoided competition by taking an idea that was already working in the US and transplanting a copycat idea to a foreign country. Incredible.

I also noticed that serial entrepreneurs founded 30% of the Unicorn companies in our data set. For this exercise, I defined serial entrepreneurs as having at least one successful previous exit. Most serial entrepreneurs in our data set had made millions before founding their Unicorn company.

After discovering this pool of serial entrepreneurs, I decided to separate them out in the next chart to see if new ideas would surface from this subset. Here is what I found:

The biggest difference was that serial entrepreneurs were much more likely to leverage their industry expertise to find their startup idea.

16 Pre-Inventive Frameworks for Startup Ideas

Over the years, I have compiled a list of 16 pre-inventive frameworks for finding startup ideas:

  1. Test, Pivot, Iterate
  2. Scratch Your Own Itch
  3. Do What You Love
  4. The Hedgehog Concept
  5. Change the World
  6. Solve a Difficult Problem
  7. Catch a Wave
  8. Copy (With Or Without Modification)
  9. Predict the Future
  10. Startup Taxonomies
  11. Find Competitive Advantage
  12. Transplant
  13. Find a Niche
  14. Find a Forgotten Idea
  15. Find Effective Surprise along the Pivot Pyramid
  16. Analogize

Let’s examine each one briefly to get a general understanding and inspire some ideas.

#1 Test, Pivot, Iterate

Concerning all acts of initiative in creation there is one elemental truth, the ignorance of which kills countless ideas and splendid plans: That the moment one definitely commits oneself, then providence moves, too. All sorts of things occur to help one that would otherwise never have occurred. All streams of events issue from the decision, raising in one’s favor all manner of unforeseen incidents and meetings and material assistance which no one man could have dreamt would have come his way. Whatever you can do or dream you can, begin it. Boldness has genius, power, and magic. –Johann Wolfgang von Goethe

It may sound naive, but when it comes to startups, Nike may have gotten it right: Just Do It.

You can’t possibly mean pick any random idea and build a startup with it, right? Actually, I do.

In artificial intelligence, some of the most powerful algorithms are iterative. They start with random inputs but quickly move to amazing results.

For example, the E-M learning algorithm uses iteration to identify clusters in data. First, it picks two random points as starting values. Second, it identifies the points most likely to cluster around those points. Third, it calculates a new more central point estimate for each cluster. Then it repeats (aka “iterates”).

Here is a picture of the E-M algorithm homing in on the two modes of eruption for the Old Faithful geyser.

The basic idea of the E-M algorithm is that if you can rapidly guess, test for accuracy, and then use the results to make a better guess, you can become smart really fast. This is the concept of iterative experimentation. It turns out, many believe that iterative experimentation is one of the most powerful ways to think about startup ideas.

You have probably heard of startup slogans like “fail fast,” “fail often,” “fail forward,” “move fast and break things.” These are all slogans to encourage iterative experimentation.

This approach is similar, in a way, to the “execution is everything” idea. Because iteration means that the original idea is very likely to change from learning and revising, execution is the only thing that matters.

Lean Startup has popularized the concept of picking an initial idea, testing it, changing it based on feedback, and repeating this process until you succeed.

Steve Blank, a founder of the Lean Startup movement, identified the 3 primary concepts of Lean Startup as:

  1. Generate business hypotheses with the intention of conducting experiments rather than spending time building complicated business plans that are likely incorrect.
  2. Build only enough so that you can conduct a test. Often you want to build only the minimum viable product.
  3. Build in sprints with lots of customer interaction along the way to ensure you are building right.

To assist this process, the famous startups accelerator, 500 Startups, developed a framework they call the Pivot Pyramid.

The Pivot Pyramid is a way to mix and match various aspects of a startup idea until all the components create a solid startup plan. If the idea doesn’t work initially, you change the elements starting at the top of the pyramid, which are easiest (lowest cost) to change. If it still doesn’t work, you keep moving on down the pyramid and iterating until it does.

Hypothetically, if you can just pivot and experiment fast enough, your startup idea will eventually click into place, kind of like solving a Rubix Cube. Pick any starting point, iterate, then go where the biggest opportunity emerges.

#2 Scratch Your Own Itch

“The idea for Dropbox was born…on a bus to New York. [Drew Houston, the founder of Dropbox] planned to work during the four-hour ride from Boston but forgot his USB memory stick, leaving him with a laptop and no code to mess with. Frustrated, he immediately started building technology to synch files over the Web. Four months later he flew to San Francisco to pitch his idea to Paul Graham of incubator Y Combinator.” —Dropbox: The Inside Story of Tech’s Hottest Startup

The “scratch your own itch” approach to getting startup ideas is to solve a problem or create delight for yourself. The logic goes something like this: if you really want a product or service, there are probably others who want it too. This approach has a storied history: Apple, Dropbox, Uber, Snapchat, Spotify, and many other companies started this way.

Scratching your own itch has another benefit. You can easily document potential ideas by just noticing unmet needs and wants as you go about your day.

In preparation for this article, I did this for a day. Here are some scratch-my-own-itch ideas I had:

  • In-store pick up is awesome when you need something fast. Wouldn’t it be great if somebody built a white label software solution for all brick and mortar stores who can’t afford to build their own?
  • Why can’t I just set up a geofence around my yard and have my dog’s shock collar keep her in that area?
  • Certain kinds of Stevia are delicious, why isn’t it in every sugary product I want to consume instead of sugar?

I can assume that if I have these itches, other people might too, and if there’s a market for it, any of these ideas have the potential to become a successful business. Let me just say “you’re welcome” in advance for these free ideas (I joke.)

#3 Do What You Love

“Your work is going to fill a large part of your life, and the only way to be truly satisfied is to do what you believe is great work. And the only way to do great work is to love what you do. If you haven’t found it yet, keep looking. Don’t settle. As with all matters of the heart, you’ll know when you find it. And, like any great relationship, it just gets better and better as the years roll on. So keep looking until you find it. Don’t settle.” —Steve Jobs from his Stanford Commencement Address

Doing what you love has the ring of a dangerous platitude. Paul Graham defines it more carefully and suggests a more practical heuristic for “doing what you love”:

“Always produce” is also a heuristic for finding the work you love. If you subject yourself to that constraint, it will automatically push you away from things you think you’re supposed to work on, toward things you actually like. “Always produce” will discover your life’s work the way water, with the aid of gravity, finds the hole in your roof.

You are likely to be consistently productive only by doing something you really enjoy. So start pushing yourself to find ways to be consistently productive.

#4 The Hedgehog Concept

“A fox knows many things, but a hedgehog one important thing” –Archilocus

In 1953, philosopher Isaiah Berlin wrote a book based on this phrase. His idea was that you divide writers and thinkers into two categories:
“Hedgehogs” who view the world through the lens of a single defining idea, and
“Foxes” who draw on a wide variety of experiences and for whom the world cannot be boiled down to a single idea

In 2001, Jim Collins applied Berlin’s framework to business. His underlying thesis was companies with a particular kind of hedgehog-like focus outperform all others.

These hedgehog companies focus on three elements, and create a business based on the common element (as seen in the graph below).

This was the framework we used to find the idea for Capshare:

  1. We were passionate about helping entrepreneurs
  2. We were very knowledgeable about cap tables, valuation theory, and venture capital
  3. We believed that we could make money selling a software service

Within these three elements, we found one common factor: a SaaS cap table solution for entrepreneurs.

The Hedgehog concept is great because it mixes elements of other frameworks we have already discussed: brainstorming, doing what you love, and using industry expertise. However, it also constrains the number of startup possibilities, so instead of brainstorming and picking an idea at random, the Hedgehog concept narrows down your options for a successful startup.

#5 Change the World

“Without a doubt, the Bible was a great source of inspiration to Columbus. He drew heavily from it and took his name, “Christ Bearer,” seriously, believing, in fact, that he had a divine mission to accomplish.” –Thomas C. Tirado about Christopher Columbus

Not many people think “entrepreneur” when they hear the name Christopher Columbus, but in reality, he was one of the greatest entrepreneurs history has ever seen, and he’s certainly one of my favorites.

But Christopher Columbus was a specific breed I call a “divine-mission entrepreneur.” His mix of unstoppable drive and certain belief that he was on a divine mission propelled him to discover a new world. Throw in a touch of crazy, and you’ve got a picture perfect definition of most influential founders in the modern world. These “divine-mission entrepreneurs” believe making money is frequently secondary to changing the world in some massively positive way, just as Columbus did.

Probably the most obvious example of a “divine-mission entrepreneur” is Elon Musk. SpaceX, Tesla, and SolarCity all started with an idea to dramatically help humanity and just so happened to turn into hugely successful (and profitable) companies.

If you have a startup idea that you think can make a massive positive impact on the world, you may be a “mission-driven entrepreneur” as well.

#6 Solve a Difficult Problem

Successful entrepreneurialism is all about solving problems. We’ve discussed this idea previously in various forms, whether it’s scratching you own itch, or having the expertise to recognize an issue in your industry that needs a solution.

It’s pretty safe to say that a significant number of startup ideas come from solutions to difficult problems or even potential solutions to difficult problems.

To create a business using this method, ask yourself if you know anybody (including yourself) that has developed a solution to a difficult problem. It’s even possible to prospect for these kinds of solutions by talking with the technology transfer offices of universities.

But founder beware. Entrepreneurs have a derogatory term for this approach: “Technology chasing a problem.” Some technologies may be novel or interesting without having that fundamental element: usefulness.

#7 Catch a Wave

“There is a tide in the affairs of men, which taken at the flood, leads on to fortune. Omitted, all the voyage of their life is bound in shallows and in miseries. On such a full sea are we now afloat.” –William Shakespeare

A former McKinsey consultant friend of mine frequently pointed out that “Why now?” might be one of the most important questions to evaluate the potential of a startup.

Timing is clearly important for startups. Perhaps the most famous example is the Apple Newton versus the Palm Pilot. The Apple Newton failed spectacularly where the Palm Pilot succeeded just a few years later.

In 1993, the technology just wasn’t quite “there yet” to make a desirable enough PDA that people would buy. It seems that the world just wasn’t ready for an all-in-one planner in the palm of your hand way back in ‘93. The Palm Pilot hit the mark just a few years later, “catching a wave,” so to speak, on the trend that Apple started.

This approach is effective if you have a nose for trends and the wherewithal to catch the wave as it’s breaking.

One of the best ways to catch a rising trend and build a business around it is to take notice of the Gartner Hype Cycle.

The Gartner Hype Cycle shows the timing of hype around startup ideas. Often there is a lot of initial hype followed by skepticism, followed by a slowly growing appreciation.

If you don’t think this happens in real life, check out this graph of Bitcoin prices.

Joseph Schumpeter talked about entrepreneurship in terms of creative destruction. This is a poetic way of describing the process of new innovation that constantly makes old approaches obsolete.

Over the last two decades, innovations in the following areas have made previous innovations obsolete:

  • The Internet
  • Social media
  • Mobile
  • AI

The internet-based (SaaS) approach of Salesforce decimated the non-web-based Siebel CRM. Social platforms like LinkedIn destroyed ACT and SalesLogix. Several mobile-centric platforms are now challenging eBay and Craigslist. And you can bet AI-led tools will start to displace those without AI.

This process of catching a wave and applying the trend to already-existing technology is a way to expand on new ideas while leaving obsolete ones in their wake. For example, can you imagine CRM without social nowadays?

Companies that are quick to capitalize on these waves can create big wins.

#8 Copy (With Or Without Modification)

“Immature poets imitate; mature poets steal.” –T.S. Eliot

If you haven’t seen it yet, check out Kirby Ferguson’s Everything Is a Remix. He makes a reasonable claim that all creativity is just copying, transforming, and combining. You can take this as a cue from Ferguson that thinking about already-existing businesses is one of the easiest ways to generate startup ideas.

The list of copied ideas in business is so large I would have to create a whole other post to list them all, and nobody’s got time for that. Instead, let’s just go with the most notorious example: Apple. While Apple is one of the most iconic innovators, it’s important to remember that its first innovations were basically transformed copies of Xerox’s computing innovations.

#9 Predict the Future

“Live in the future, then build what’s missing.” –Paul Graham

If you’ve ever wished you could see the results of a ballgame or the success of a certain company so you could ride the cash wave, you’re not alone. But if you’re an entrepreneur, chances are you’ve taken it a step further, imagining what innovations the future held so you could just build it yourself.

That is just what Paul Graham thinks the best entrepreneurs do. They are living in a future where the problems they face everyday already have solutions.

#10 Startup Taxonomies

In business school, I started an effort I called the startup genome project where I tried to categorize startups the way Pandora categorized songs–building genres and subgenres.

The ultimate goal was to create a taxonomy of startup idea themes. As you can imagine, this was a massive undertaking. While I did not complete the taxonomy, I created a fairly comprehensive list of categories.

The basic idea of startup ideas taxonomies is that by coming up with categories of startup ideas you can quickly identify opportunities for new startups in unfilled categories.

Eric Stromberg recently published a brilliant startup taxonomy tool he calls the Startup Idea Matrix:

Eric takes a set of startup themes and puts them across the horizontal axis of a matrix with industry sectors on the vertical axis.

We can expand on this idea a lot and build several different startup idea matrices:

  • Startup themes by industry (this is Eric’s matrix)
  • Startup themes by geography
  • Startup themes by function (financing, marketing, HR, etc)

There are a couple of awesome things about these frameworks:

  1. It makes it really easy to see what has already worked
  2. It helps you find some easily plausible startup ideas
  3. It also helps you identify startup ideas that don’t yet have a lot of competition

#11 Find Competitive Advantage

This is probably the most underrated startup ideation approach, especially when you realize the, well, advantage of competitive advantages. When you’ve got this, you don’t even need the golden gem of conventional wisdom: novelty.

As we’ve seen in the idea source charts, most successful startup ideas have come from industry expertise. If you have the expertise to start a startup where you have a competitive advantage, you probably have a pretty good shot at success. Also, if you can execute and differentiate, then novel ideas will probably come. This explains a big part of serial entrepreneur success.

#12 Transplant

As we mentioned above, it is possible to succeed with a non-novel idea in competitive market even if you don’t have a competitive advantage.

Here is a list of Unicorns with “obvious” ideas–many of which fit this description:

  • Didi Chuxing (Uber in China)
  • Flipkart (E-commerce in India)
  • Dropbox (50+ direct competitors at founding)
  • Infor (many competitors for cloud-based middleware)
  • Spotify (many competitors including iTunes and Napster)
  • Zhong An Insurance (web-based insurance in China)
  • Snapdeal (E-commerce in India)
  • Olacabs (Uber in India)
  • Coupang (Groupon for Korea)
  • Slack (many work-focused messaging competitors)
  • Royole Corporation (display manufacturer)
  • ContextLogic (E-commerce in China)
  • Domo (many business intelligence competitors)
  • The Honest Company (consumer products)

Are you noticing a theme here? A huge percentage of the obvious startup ideas are what I call “transplants.” This is a form of Copying–but with a twist. You take an idea that has worked in one geography and transplant it to another.

In the list of obvious startups above, the direction of the transplants was often from the US to other countries. But Starbucks famously started when the founder transplanted Italian barista-style coffee to the the US and then to the world.

This works because most startups simply don’t have the resources to fully serve every potential customer immediately. If you can spot an opportunity like this, you could easily see transplant startup success.

Transplanting is brilliant because it takes an obvious idea but circumvents startup EMH by going to an area where competition is low.

There is more than one way to transplant (though geographical transplanting is the most common):

  • Industry transplanting. Taking something from one industry to another. Like moving assembly lines from cars (Ford) to hamburgers (McDonald’s).
  • Generational transplanting. Taking something from one generation to another. Like emoji usage from kids to adults.
  • Functional transplanting. Taking something from one function to another. Like using conversion optimization for back-end UX.
  • Group-to-group transplanting. Basically anything you can transfer from one group to another. In the early days of online dating, several copycat sites cropped up to target different groups: Christian dating, Muslim dating, etc.

#13 Find a Niche

Starting a business on a niche idea has an obvious advantage: there is little to no competition. But the drawback is that you are unlikely to ever raise venture capital if you go after a niche market, so says that pesky conventional wisdom. In this case, conventional wisdom would be correct; niche markets get ignored by big money.

However, a lot of businesses start without any consideration to market size, and some even discover that their market is much larger than they had anticipated. Even if massive demand doesn’t appear out of the blue there’s no problem with creating a niche product for a niche audience of people willing to buy.

#14 Find a Forgotten Idea

The founder of a company called Kozmo.com was in my business school class. The premise of Kozmo was to get people to pay for the delivery of local products or services like food. Kozmo died in the web 1.0 internet bubble.

Another web 1.0 blow up was Webvan. It was a web-based grocery delivery company.

Do these ideas sound familiar? They should. Currently, there are so many food delivery startups that TechCrunch has a dedicated channel for all of the related news. Several are on the Unicorn list.

As previously noted, timing is essential. Just because an idea has failed, doesn’t mean it isn’t a good one, it may just have come at the wrong time. And just because the world has forgotten it doesn’t mean you should too. If you pinpoint an idea that has failed because of timing, file it away for later use. You might find that you can start it up later with a twist that makes it a success.

#15 Find Effective Surprise Along the Pivot Pyramid

For startups, many effective surprises occur in one of the categories of the Pivot Pyramid discussed earlier.

Let’s look at a few examples of startups that found novelty within the pivot pyramid.

Airbnb

Homeaway existed for nearly a decade before Airbnb, and the products are essentially the same. So why did AirBnb still succeed even with such a big competitor?

Because Airbnb pioneered several new ideas:

  • Letting people rent any amount of space
  • Letting people rent in non-vacation homes
  • Letting people rent whether the owner was there or not

HomeAway focused only on vacation rentals of owners’ second homes while they were away.

Using the Pivot Pyramid, Airbnb did several effectively surprising things:

  • New customers. Airbnb targeted anybody with a little extra “floor space” not just rich second-home owners. It also targeted people willing to rent in nontraditional ways.
  • New problem. Airbnb allowed people to rent homes even in non-vacation-home areas like big cities
  • New solution. Airbnb offered a much lower-end product (i.e., renting a couch)

If you were like me at the time, I found all of this quite surprising. I wouldn’t have guessed you could create a billion dollar company based on the idea of renting couch space. But perhaps you could have predicted it by testing different combinations of the pivot pyramid.

Snapchat

There have been literally thousands of picture-driven social networks that tried to reach the level of success seen by Facebook, Instagram, and Pinterest. Most of those have failed… but Snapchat didn’t. Why were they an exception?

Again, using the Pivot Pyramid, we can see Snapchat did a lot of surprisingly effective things:

  • New problem. People wanted to share things about themselves but didn’t want to create a permanent record of it. Snapchat tackled this problem.
  • New solution. Snapchat’s technology allowed for users to make pictures ephemeral–disappearing after a few seconds. They also added surprising mini-features that appealed to a younger demographic. Who knew we wanted to see pictures of friends barfing a rainbow?

I personally found it surprising that in a world crowded with social media, yet another one would succeed–especially when it’s only major differentiator was a feature that made its pictures disappear. Again, who could have predicted that an idea as simple and easy-to-build as disappearing pictures would create a multi-billion dollar success?

Essentially, if you can create effective surprise in just one of the following areas–customer focus, the problem you are solving, the solution, the technology you are using, or your growth strategy–you can often create not-so-surprising success.

#16 Analogize

Another way to find effective surprise is to think about startups using analogies.

From my experience, analogical reasoning is one of the most common ways founders get ideas. After all, who hasn’t had a conversation with a founder that went something like this:

Person 1: What’s your new startup idea?
Founder: It’s Uber for laundry.

There are a number of somewhat silly startup idea generators that use this basic idea of conceptual blending to randomly create startup ideas. But in truth, this formula of analogical reasoning can actually be quite effective. It has the essence of effective surprise, and uses the idea of transplanting – in this case from one industry to another.

Applying It to Your Expertise

It’s important to note that the biggest category of Unicorn ideas was the one based on industry expertise, not novelty. This category accounted for 38% of serial entrepreneurs’ success and 31% of all companies’ success.

As we’ve discussed above, industry expertise can give you advantages in just about every way a startup can succeed. Applying any or all of the pre-inventive frameworks to your own areas of expertise is the quickest route to startup success.

Leveraging industry expertise into coming up with great business ideas is such a big topic it will have to wait for my next article. In the meantime, apply all of the concepts we have discussed above to your own business and see what you come up with.

Remember, it may feel somewhat blasphemous, but just starting your own directly comparable business as a competitor to your current business is also one of the most tried-and-true startup success approaches.

Conclusion

Novelty captures our imaginations like nothing else does in the world of entrepreneurship. After all, entrepreneurs are creatives, and resisting the urge to create something no one has ever heard of before is like resisting the urge to check your company’s growth analytics. Entrepreneurs the world over continue to agonize in business school cafes, lonely hotel rooms, friends’ apartments and in small cubicles of big companies about finding the next great novel idea.

The truth is, novelty is overrated, although not in the way you might think. Some of the biggest and most well-known startups have highly novel ideas. They capture our collective imagination, and inspire us to create the next big thing. But these startups are in a very small minority.

Startups are as diverse and varied as the apps on your smartphone, and come from a multitude of backgrounds and ideation strategies. Narrowing your focus to only novel ideas might actually be restricting would-be founders and decreasing the likelihood of finding a great startup idea. Novelty becomes even more irrelevant in the face of excellent execution and competitive advantage. You would actually do better to source ideas, not from that intangible realm of novelty, but from industry expertise, scratching your own itch, or copying.

Use these pre-inventive frameworks like blueprints that you can mold and adjust around the solid foundation of your industry expertise. Execute well, continue to evolve, and you have a shot at achieving that highest of entrepreneurial goals: startup success.

Good luck and let me know if you have any questions or comment!

Special thanks to Ariana Crisafulli and Ian Artinger who were both major contributors to the content of this article.