Prediction: Idea Management will grow 3X in the 2020s
Sustained economic growth is central to modern civilisation. Ideas, i.e. invention and innovation, are tightly coupled to economic growth.
But ideas are getting harder to find.
It takes ~5% more effort per year for idea inputs (researchers and research dollars) to translate to idea output. This trend is persistant, global and consistent at both micro and macro scales.
Funding is finite.
The incentives then are aligned at the economic level and within the ideas industry (universities, R&D centres, funding agencies) to produce more with less.
There is an entire under-discovered industry of idea management that has yet to emerge to help teams, individuals and institutions to find more ideas, faster and to squeeze more out of them when they do.
My prediction is that this idea management industry will grow more than 3X in the coming decade.
Ideas are getting harder to find
Ideas are getting harder to find.
Across industries, products and organisations, research effort is rising while research productivity is declining sharply. Since 1930s, in the US research effort has risen 23X with its productivity declining by 41X. Research productivity is falling 5.1% per year.
This thesis has been widely examined in the economics literature at the macro-level. Growth rates are relatively stable over time, while the number of researchers has risen enormously. In short, research productivity has plummeted in virtually every place we look.
In other words, to get the same ultimate output (economic growth) we must put in more and more research input.
In 'Ideas are Getting Harder to Find', Bloom et al. examine whether this declining relationship between research and economic growth holds true for research outcomes also. Is this a problem of finding ideas or a problem of converting those ideas to economic growth?
The problem is finding ideas.
The paper focuses on the relationship between 'research input' and 'idea output' for semiconductors, agriculture and medical research. In each case, research productivity drops.
It is ~18X harder today to generate the exponential growth of transistor density behind Moore’s Law than it was in 1971 [1971-2015].
Research productivity declines by 4X for agricultural crop yields. It takes 4X the researchers to grow crop yields at a constant rate [1970-2007].
For medicine, it is 5X harder to convert research (publications, clinical trials) to save a year of life across a range of illnesses [1950-2014].
At the macro-level, we know from the literature that we need to double research efforts every 13 years to maintain economic growth rate. This is clearly unsustainable.
We now know that research productivity, i.e. finding ideas, is where at least some of this bottleneck is.
Better ways of finding ideas is more important now, than it ever has been.
— — —
Assumptions
Ideas are different from other goods. An idea can be used simultaneously by multiple people.
Simultaneous use of a new idea means exponential increase in research can lead to exponential economic growth.
Sources & Inspiration
Ideas are Getting Harder to Find, Bloom et al. 2020
David Deutsch on Possibility in Science
Science often predicts – and brings about – phenomena spectacularly different from anything that has been experienced before.
For millennia people dreamed about flying, but they experienced only falling.
What dreams of flying will soon be made real and put an end to our falling?
Ways of Exploring II
Selecting stepping stones is the path to future ideas.
Building on the compasses proposed by Stanley and Lehman, I propose four of my own.
Energy-giving. We are energy-harvesting phenomena. This is true at an evolutionary level and at a societal one. More importantly, it is true perceptually. We like the feeling of having energy and are subtly drawn to the things that give it to us.
There's a practical side to this; energy-giving is a good measure of creator-idea fit. The 'energy' that we feel is a signal that there is something about this idea that is resonant, inspiring. It is breathing energy into us, making us want to act, explore and tinker. Tactically, action creates more action. Stepping stones in practice aren't fixed things. We are more likely to uncover the next stepping stones through action and action requires energy. Use these pockets of energy, these momentum stepping stones to propel you forward, ad infinitum.
Narrative Quality. Agency is under-rated. We desire to be the main-character. We like to articulate our own lives. The narrative arc we are stumbling through is powerful, momentum-giving stuff. Breathing meaning into our decisions; elevating them beyond mere cold, empty atomic interactions in space-time. What stepping stone has narrative quality - that adds to your own adventure, making it bigger, more resonant, more captivating.
If the stepping stones you seek are rock-solid truth in a scientific sense, this may be a poor compass. But for other domains, 'what makes for the better story', is a pathway worth following. In Ireland, an artefact of the Celtic story-telling tradition has left this etched on the psyches of many - "I did it for the story".
Other-worldliness. These are the things that are beautifully strange. That speak to us from beyond the veil. As if from another planet, dimension or deity. This other-worldliness sits in the idea-space beside awe. Surfing wonder to wonder, is a life well-lived. It holds clues to our profound creativity and curiosity. And will lead to paths even more other-worldly still.
Deep links to the past. We were not born yesterday. We stand at the end of vast, intricate sequences of events, situations, ecosystems and decisions about which we know very, very little. And yet in these winding infinite pathways into our deepest past somethings remain constant. Some things hold true. Some things have stood the test of time. Stepping stones that have this quality engrained within, that have those etched upon them branch to stones that continue this quality. They are likely to be Lindy themselves. We not only seek novelty. But novelty that lasts.
Choose your compass.
Ways of Exploring
Ideas are stepping stones to more ideas.
Selecting stepping stones then is of utmost importance. How we select these stepping stones matters. We must choose our compass.
In Greatness Cannot be Planned, Stanley and Lehman propose a set of compasses worth considering.
important clues we can consider: inspiration, elegance, potential to provoke further creativity, thought-provoking construction, challenge to the status quo, novelty, analogy to nature, beauty, simplicity, and imagination.
Let's take these apart. Let's examine a few.
Inspiration is a two-fold thing. A measure of how drawn we are to a topic. But also a proxy measure for how generative the idea is likely to be. What faint outlines of future stones are sparked from this one.
Elegance points to an underlying beauty; a light-quality imbued within an idea's structure. Elegance has symmetry and simplicity; it is more likely to be generalisable than something cobbled together and clunky. We are drawn to beauty and should trust it. Beauty begets beauty.
Analogy to nature. There is 4 billion years of evolutionary experimentation evoked and encoded within design in nature. Ideas, inventions and directions that are in alignment with nature - that are biophilic - have elements that have made these designs successful. These Lindy designs have Lindy underpinnings; they have lasted the test of time. We know from structures in nature that these designs will have natural evolutionary steps forward too.
Simplicity. All complex systems arise from simple systems. Simplicity is generative. Simple things are useful components of more complicated things. Simple patterns are easily replicated across domains.
Potential to provoke other creativity. Most of us have an intuition of what is generative. What are the ideas that lead to more ideas. Some ideas just drip of that potential. Those ideas are alive almost; pulsing with a vital force of their own. Waiting, rattling to be unleashed. Follow those ideas. Take those paths.
Choose your compass.
Everything, everywhere is always moving. Forever.
Everything, everywhere is always moving. Forever. Get used to it.
This harsh wisdom perfectly sums up the realities of competition. Delivered by Logan Roy to his wannabe C-suite daughter in Succession.
The more competitive the environment, the less you can rest upon certainty, safety, and security.
More things move. And they move faster.
Perfectly sums up life in Tech middle and senior management. More so, now as the VUCA-ness increases.
Excellent reminder for those wishing to excel at Chinese Tech companies.
Sol LeWitt on Conceptual Art
Ideas can be works of art; they are in a chain of development that may eventually find some form.
All ideas need not be made physical.
Literally true of digital creations. Truer now with the accelerating emergence of Generative AI.
Where ideas apparate from imagined prompts and whispers. The chain of development taking place on GPUs in abstract neural nets.
And the idea of conceptual art sits somewhere between the grey neurons of the artist and the neural net of the silicon.
Sources & Inspiration
Sentences on Conceptual Art, Sol LeWitt.
VUCA by Design
The world of 02022 is volatile, uncertain, complex and ambiguous. It is VUCA.
TikTok is VUCA by design.
Mirroring these 21st century realities, a company can better adapt to its defining characteristics. Encouraging this VUCA-ness to grow within the organisation and indeed nurturing VUCA has benefits.
VUCA environments can spark creativity and innovation. Team members are forced to rethink old patterns and adapt to emerging complex problems. Out-dated ways-of-working are never solidified or cemented.
A VUCA organisation is designed to be flexible and adaptable. No space is moving so quickly at scale as consumer internet. With the newer platforms having profitable but short shelf-lives. TikTok understands this. It does not want to be a flash in the pan, but a lingering presence and ready for the 2020s, 2030s and beyond.
A VUCA organisation is designed to be a learning organisation, where team members must continually develop their skills and knowledge. At TikTok this learning is mostly done on the job. It is learning that must match the pace of the company in out-pacing the market.
Adapting to the pace and VUCA environment of TikTok is the biggest challenge for new hires.
It is also the most exciting part of being part of that environment.
— — —
This post is part of the TikTok Sequence, a series of short thoughts inspired by my time working with TikTok. The posts focus mostly on the experience of work culture at TikTok and companies like it.
Idea Maze, Multi-Maze
The Idea Maze is a company's path to its desired outcome.
A path that traverses the limitless series of pitfalls and obstacles that lie before any venture worth undertaking.
Balaji Srinivasan puts it like this:
"A good founder is capable of anticipating which turns lead to treasure and which lead to certain death.
A bad founder is just running to the entrance of (say) the “movies/music/filesharing/P2P” maze or the “photosharing” maze without any sense for the history of the industry, the players in the maze, the casualties of the past, and the technologies that are likely to move walls and change assumptions."
Strong ideas are well-considered, multi-year plans that consider many possible paths through an unknown, changing world.
In this path of paths, there are near-infinite degrees of freedom to choose. And therefore, near-infinite ways in which to be wrong.
At the beginning, a founder* is walking up to the edge of several mazes. Peering in, perhaps taking a few tentative steps to get a feel for how the maze might work and how they might behave in any maze.
Now founders must choose. There's little benefit in a series of hesitant steps at the mouths of many mazes.
With a maze selected, we've passed the initial hurdle. The maze is ready to be mapped out and meandered through. A founder can focus on this idea maze, and this maze alone.
Beware however, some mazes open out into others.
They intersect and cross. You can traverse them.
Consider the idea maze then as a multi-entry, multi-exit maze with occasional intersections between different mazes. And many more intersections between strongly overlapping idea spaces. These intersections are essentially minor-pivots.
Now, some mazes overlap and intersect in ways that are surprising. Before Uber, few would have thought that the taxi maze had passages across to the scooter maze. Or that there was an off-shoot into Food Delivery or even double-sided ratings mechanisms.
Most paths in the idea maze fork to dead ends, traps, doldrums and monsters.
And as in the single maze - the bad founder who rushes to the entrance of many mazes tends to traverse many contiguous mazes. Some traversing will be necessary as assumptions and the world we live in change. But many traverses simply leaves you deep within a maze you didn't expect to find yourself in at all. And one for which you are ill-prepared.
With little sight out of it.
A new maze full of unfamiliar traps and unexpected monsters. And you with a shrinking set of ideas of how to move forward.
Consider then, each traverse (each pivot) an almost-reset. Just because you're deep in a maze - doesn't mean you're deep in the contiguous one.
Navigate with caution.
Remember, mazes are multi-dimensional and moving.
— — —
Notes
*I am using founder loosely here to include those starting new ventures within existing companies and institutions.
How are you Navigating?
Once your time here is done, you will have carved a path through the space of possible paths of work. A path of what you worked on and with whom. A path whose contours will be the investments you made in energy, focus, devotion and hopes.
This path will be but a partial interpretation of your life.
But now, as in the past, it is important how you spend your time, how you use your brain and body, and what the accumulation of your energies might look like.
So what will that path look like?
And how much influence will your work have in shaping its course? That is a question that you are answering right now. It is a question that you have more of a say in, than you may have assumed.
Understand this. When your life is over - and should someone choose to think of such material things - your path will have been fixed.
Beyond you, in smaller part, that path will continue on in the lives of others. Through influence, or through descendants. Fading, and dispersing the farther we move away in terms of relationships and time.
The decisions you took this morning and will take tomorrow determine the path, its course, shape and turns taken.
So how are you navigating?
Stepping Stones in Idea Space
Ideas are stepping stones to more ideas.
Some ideas are useful.
String enough useful ideas together and you may find yourself in an unexpectedly interesting and lucrative neighbourhood.
In Greatness Cannot be Planned, Stanley and Lehman compare ambitious projects with generative art. In the search space of ideas, we are mostly limited to ideas that are close to ones we have previously encountered*.
For highly ambitious objectives, i.e. ones which require multiple innovations or inventions, the path of stepping stones from here to our goal is definitively obscured.
It is like looking across a foggy lake, with only our immediate next steps visible.
How then are we to choose where to place our feet? Which stepping stones to choose.
Stanley and Lehman, critique our bias for objective measurement to evaluate these stepping stones when our destination is obscured in the fog.
If the rules that govern search algorithms can be generalised to ambitious goals then this is a compelling rationale.
Regardless, there is a lesson here. A reframe of how to evaluate any next step in a path into the unknown.
Be that of constrained problem-solving, invention or indeed in deciding on what to work on.
— — —
Notes
*either directly or second-hand.
Sources & Inspiration
Why Greatness Cannot be Planned by Kenneth Stanley and Joel Lehman
Org challenges and UX of AI
You ship the org chart.
More precisely, the shape and composition of AI teams directs what Product ships.
Given the relative newness and importance of AI, the organisational challenges in the UX of AI are key to shipping better AI Product.
Outside of the major tech companies in particular, I’ve observed three situations that are pervasive and strongly impacting what gets shipped.
A large knowledge and cultural gap between UXers and MLers. UX designers are unlikely to have a strong ML background and ML practitioners are unlikely to be expert at UX.
The concept of AI products are likely to come with organisational biases (positive or negative).
AI products are often not independent within the org. Org boundaries impact what can and will be shipped and how much of the UX we can control.
How I got into AI
It started with a video of a monkey moving a mechanical arm with its mind.
I saw it as a teenager on Horizon, as part of an introduction to Brain-computer Interfaces by Miguel Nicolelis’ research group at Duke. Here’s a more recent version:
That planted a seed. Working in neuroscience, or signal processing in neuroscience became a sort of passive, background goal.
The seed lay dormant until, Liam Marnane, one of my lecturers at UCC was leading a research project on detecting brain seizures in newborn babies. They were using Machine Learning to do it. That was the portal to neuroscience first and then AI.
I joined that research group to do a PhD in ML applied to epileptic and neonatal seizures. And then moved into industry as a data scientist and quickly moved across to AI Product Management from there.
What is the UX of AI
UX of AI is the experience that a user has while interacting with a product that uses artificial intelligence.
In other words, the experience of interacting with a product that is built with computer software that learns and adapts (or has learned and adapted) without the help of explicit programming.
UX of AI is increasingly important as most of today’s internet products have at least some AI built in (and more and more hardware products are using AI) and the experience of interacting with AI products can be fundamentally different than with traditional software products.
Designing AI products with UX top of mind can lead to a fundamentally better experience for the user, and is more likely to lead to good product outcomes for the PdM or designer.
Podcast: UX for AI Products
Thoroughly enjoyed speaking about the UX of AI Products on the PM Hub Podcast in 2021.
In the conversation we covered:
What is UX for AI products and why it matters.
How UX of AI changes for different product types.
UX of AI Strategy
Links to the various podcast platforms found here.
TikTok Time
Time behaves differently in speed-obsessed environments.
It warps what it feels like to 'be'. And it reconfigures in a brute-force way your assumptions about what's possible.
In a smoky bar in East Berlin, a Product leader at Zalando confided that 1 year at Zalando was the equivalent of 5 years in a typical company. I was full of beer and I was skeptical. Later, finishing a 2 year stint at Zalando, I concluded it was the best 10 years of my career.
Time, spent in the right environment, can manufacture experience at rates that are unthinkable in most companies. The simplest explanation is that operating a faster clock speed means you compress more examples into a fixed period.
There is truth to that. But there is also something to be said for the sheer pace, changing the experience itself. It's like playing tennis with opponents who are multiple levels above you; who hit big, at speeds you struggle to handle. You are forced to play out-of-your-skin; on instinct. And if you can adapt to that pace, you can close the skill gap very quickly indeed.
TikTok is the most extreme version of this analogy I can imagine.
A sort of multi-threaded Zalando, where business plans are birthed in hours and all's changed utterly in a week.
— — —
This post is part of the TikTok Sequence, a series of short thoughts inspired by my time working with TikTok. The posts focus mostly on the experience of work culture at TikTok and companies like it.
Above All, Speed
Speed is the defining characteristic of work at Tik Tok.
Speed is the meta-value that informs everything else.
In doing 100s of interviews for TikTok, I found myself answering an unusually broad range of interviewees’ questions about the company with that one-word answer. What's it like to work at TikTok? What are you most surprised about working at TikTok? What's best and what's worst? How is the culture?
Ultimately, it all boils down to speed. Working at TikTok is faster than anywhere else any of us have worked. Speed is the Principal Component that explains almost everything else about working at TikTok.
That dedication to moving at extreme pace has both predictable and surprising downstream effects.
On the human side, rather predictably, it raises tensions and pressure between individuals and teams. It removes the finer points of change management.
Surprisingly, it provides some inoculation against politiking. Org borders ebb and flow. Things are moving too quickly to deliberately* be a prick. We just need to get things done.
As a philosophy, everything boils down to a commitment to Ready-Fire-Aim. Do it, ship it. With enough extra cycles from removing planning and inertia, we will tweak and fix mistakes. And learn more in the process.
This is par for the course for startups. At TikTok-scale it is remarkable.
Philosophically, there is good reason to suggest that emphasis on speed (and hence quantity) leads to better outcomes. We know this about pot-making. And we know that, fittingly, in machine learning, quantity has a quality all of its own.
Speed has a quality all of its own.
This obviously takes getting used to. For deeper thinkers, and for HSPs in particular, this can be especially jarring on preferred ways-of-working and directly on the nervous system.
But it's exciting. And addictive.
And an opportunity to interrogate assumptions about ways-of-working. To get comfortable working with a torrent of ideas and updates from the hive mind. To 'Yes And' more than not.
To 'Yes And' quickly.
It’s an early glimpse of where the world of tech is going. More than anything, I believe that TikTok is a mirror of where the tech world is racing toward. Looking ahead to AI and Web3 in particular - will the average pace of development stay the same, slow down or speed up?
The defining characteristic of the future. Above all, speed.
— — —
This post is part of the TikTok Sequence, a series of short thoughts inspired by my time working with TikTok. The posts focus mostly on the experience of work culture at TikTok and companies like it.
— — —
Notes
*People will still be people.
Neri Oxman on Innovation and Loneliness
"If you can measure it, it doesn’t count.
You know you’ve reached innovation when you’re lonely.
You know that you’re doing something new, pushing boundaries, when there’s no one or nothing against which to compare yourself.
Experimental design entails failure."
From a conversation on the Future of Design with Norman Foster.
Human Nests and ‘Good Enough’ Design
We used to sleep in nests.
All the great apes make nests. All, bar adult gorillas, make them in trees.
The nests of our hunter-gatherer ancestors were ground nests. Half-dome shelters of grass, leafy twigs and branches. These temporary structures were used for resting or sleeping as long as the group stayed in one place and abandoned when the tribe moved on.
In The Old Way, Elizabeth Marshall Thomas, describes how these nests offer 'just enough' protection for the Ju/wasi from big cats.
A flimsy nest doesn't look like much protection from a leopard or lion but it addresses the biggest part of the problem. Put simply, it is 'good enough' design.
Cats prefer the rear approach, first studying the victim briefly, then taking him by surprise from a position of advantage. But normally the cat has just one chance to do this and must see the victim clearly from the start. Hence to break through a tshu in hopes of catching something inside is not efficient - there you are with a mouthful of grass while your intended prey wriggles out of your paws and goes scuttling off sounding an alarm and bringing its fellows swarming around you with flaming branches, stones and noise. Your hunt is ruined.
And so when looking to 50,000 years of design ancestry, we can conclude that our design habits are those of making temporary, partial, disposable structures.
Design that does just enough to solve for the threat of our largest predators.
'Good enough' design is design that lasted.
Artist impression from Midjourney.
Sources & Inspiration
The Old Way, Elizabeth Marshall Thomas
The Pain of Convergence
Any finite creation*, even the very best imaginable, is a process of continuously waving goodbye to possibility.
The path from imagined to real is an ever-shrinking sink of possibilities. Whittled down until only one possibility remains.
That long goodbye is a battle with friction, with distraction.
Charles Eames famously focused only 1% of his energy conceiving a design. The rest? Holding onto that design as the project ran its course.
More, that final moment of completion is a moment of loss. The loss of all the other forms this imagined creation may have taken.
No wonder we procrastinate and start new things.
Notes
*This is true of all creativity. From the meta (futures, product vision) to the minute (features, lines of code).
Sources & Inspiration
Four Thousand Hours, Oliver Burkeman
Art & Fear, David Bayles & Ted Orland