Does Human Knowledge Double Every 5 Years?

The Outcome Fallacy and the Limits of Desktop Research

Throughout this blog, I look for positive outcomes and improved results coming from increased human knowledge regarding big problems. I rather blithely say 'if we have doubled our knowledge regarding cancer, why aren't survival rates doubling?", and things like that.

But there is no reason to assume that this should be the case, and I know that. It could be that for a problem as difficult as cancer, a doubling of human knowledge will only produce a 10% improvement in survival rates. I don't know. I don't think anybody else does either. (For God's sake, if you do know, tell me.)

Intractable problems may not have a solution at all. Or we may need to multiply our knowledge 100 times before we can solve them.

But looking for a dramatic improvement in a problematic situation seems legitimate--and we have already found one specific instance where human knowledge has dramatically improved the outcomes of an intractable problem--battlefield survival rates.

So, fair warning. I'll continue to use language loosely and describe human knowledge and outcomes with only the metrics that Google proffers. On the other hand, if this blog ever starts gettng traffic, you will find me flexible about ways of improving both my language and my metrics.

In a Knowledge Economy, Is Global GDP a Proxy for Knowledge?

Let's assume for a moment that it might be. World GDP has been growing strongly of late--above trend, and in the face of challenges that formerly may have caused it to falter. But it has not grown at anything that would resemble a doubling every 5 years--in 2000, according to the World GDP was 32 trillion US dollars, and in 2005 it was 44.4 trillion. Great growth--not doubling.

You could point out the obvious--that knowledge is not spread evenly throughout the world. But the part of the world where knowledge might be presumed to be greatest grew at a slower rate than the part of the world where it might be presumed to be lesser. No doubt much of that growth is due to increased human knowledge. But more than a little was due to more workers.

Hmm. This would not be the first time that we see indications that the more dramatic effects of knowledge occur when existing knowledge is made available to new people. Making 30-year-old call centre technology available to India, disassembling factories in England and reconstructing them in China and Indonesia, this is knowledge transfer, not creation. And, to repeat my refrain, this is great, good and wonderful--if you don't have an agenda that demands exponential growth in new knowledge.

The search continues.

Why This is Important (I hope)

I don't fear death. And I won't fear death--unless and until there is an alternative.

A group of scientists and science popularizers are creating the expectation of an alternative to death. It's a pretty important concept, and a powerful hope to offer. The account I've read most recently is by Ray Kurzweil, author of The Singularity is Near. (If you're shopping from the UK, as I do, click here.) Kurzweil, as well as many of the other popularizers of the exponential growth of knowledge, argues that... well, something is growing so fast that within the lifetimes of many of us, machine generated knowledge will sort of take over from us. I'll be talking a lot about that 'something' in future posts.

For now, let me just say that if human knowledge is not doubling quickly enough to give exponential growth rates, a lot of what these people predict kinda falls on its face. Now, I want this future to come to pass--it holds too many potential benefits to ignore or wish away. (I do wish, however, that a more manageable version of this idea was available). But it all depends on the exponential growth of human knowledge.

Now, these people (and I'll be naming who I'm talking about later) talk a lot about exponential growth in information technology, etc., and I can well believe that the price of computing and storage is dropping exponentially. In fact, I know it has from personal experience. But having free memory (stored on a free weblog server, for example) and infinitely fast access to that memory does not mean a blessed thing if I don't have useful information to store and/or access.

So what's at stake is our future--gee, that sounds somewhat dramatic. But a lot of people who read these Singularity/Immortality/SENS/etc. protagonists need to know if this is science or science fiction. I'm one of those who need to know. That's why I'm doing this. Just so you know.

non-Conclusions on Medical Knowledge

After writing the previous three posts, I have some thoughts about human knowledge and medicine. I reserve the right to change my mind on any of this:

• Human knowledge about matters medical has increased dramatically, but does not double every 5 years
• Medical outcomes have increased human health and longevity, but these outcomes do not 'follow the money,' nor the research
• Managing newly acquired medical knowledge is now the key issue impeding further progress

The second hypothesis I am testing is whether or not the increasing availability of knowledge to a wider population can satisfy the criteria of doubling human knowledge. I believe that medicine may offer strong evidence for this.

The invention of the polio vaccine did not eradicate polio. It was the diffusion of the knowledge about the potential of the vaccine, first to doctors, then to mothers, that made the difference. Nobody has found much of a cure for lung cancer, but in areas where public policy has pushed awareness of the link between smoking and cancer (and social activism has become more militant), smaller percentages of the population continue to smoke, and this has had a beneficial effect.

The same is roughly true regarding the benefits of mammography, checking of the colon and prostate and pap smears. It is pushing the knowledge to the people that has made all the difference. Human life expectancy has risen in the developed countries to between 75 and 84 years of age. One hundred years ago, it was between 42 and 48. Almost all of the difference seems to me to be due to increases in human knowledge, but completely conditional on this knowledge being published and affecting the behaviour of the general public.

The War on Cancer

There are many who say that we are losing the war on cancer. This article (it is from Fortune, reproduced in PDF format on the Aperio blog), offers depressing information. It says (in part)

• The age-adjusted percentage of Americans dying from cancer is the same now as it was in the 1970's... and the 1950's
• That 33 years ago, half of all cancer patients survived 5 years or more following diagnosis. That figure has climbed, but to only 63%
• Most of the progress that has been made has been due to early identification of the disease
• American spending (all public and identifiable private sources) on cancer research amounts to $200 billion since 1971
• The cancer research community has published 1.56 million papers just on signaling pathways and its related genes

And yet... for all the article's accounts of false starts, corporate greed and bureaucratic fumbling, a great body of knowledge has been accumulated. A lot of wrong paths have been followed--but if they are identified and no more time and effort is wasted on them, it means something. Those with many years of experience in the field are starting to talk about drug cocktail approaches that target both the tumor and the environment it lives in. This is after watching AIDS treatment make spectacular rapid progress while cancer treatment seemed stalled. There is talk about imitating strategies that were successful in the rapid reduction of heart attacks and strokes, as well. And again, a lot of basic research has been done.

Knowledge has been created. Outcomes have not matched this creation. Politics and corporate policy have interfered. But it looks... possible... that when the politics and corporate policy are finally removed, that progress can jump and outcomes catch up. One hopes.

Cancer Survival Rates

Again, a doubling of human knowledge should result in different outcomes in areas where that knowledge is applied. A lot of money is spent in improving our knowledge of certain cancers. If we double our knowledge, treatment of cancer should have a beneficial effect--improved survival rates being one effect we can measure.

A Google news search on cancer survival rates shows evidence of this. From Canada, the National Post reports that "the death rate for breast cancer for Canadian women has fallen 25% since 1986, when organized breast screening programs began being rolled out across the country." Apparently, this reduction is down to human knowledge from the early 1980's (beneficial effects of mammography) being dispersed through the wider population. Again, to me this counts. This is good news, great news. But again, if we keep a 5-year clock ticking, this news will disappoint, as it shows nothing remotely indicating a doubling of human knowledge within that timeframe.

Similarly, MedicineNet says, "Statistics (released in 1997) show that cancer patients are living longer and even "beating" the disease. Information released at an AMA sponsored conference for science writers, showed that the death rate from the dreaded disease has decreased by 3 percent in the last few years. In the  1940's only one patient in four survived on the average. By the 1960's, that figure was up to one in three, and now has reached 50% survival." So we see a doubling of the beneficial effect, but over 60 years, not five.

More encouragingly, St. Jude's Research Hospital shows 5-year survival rates for some cancers since 1962, and some of them might just qualify as 10-year doubling (pause to experiment on how to present this information... Okay, got it--click on the image to enlarge to readable size).

Again, a miracle, but it doesn't fit a 5-year window--it's 10 years for the best performing cancers.

Improvements in Medical Knowledge

One of the problems in doing a weblog such as this, based on desktop research that is basically powered by Google and supplemented by Wikipedia, is that one can only work on work that has already been done.

If I want to tell you about the doubling of human knowledge in medicine, I have to look at something that has been measured, and measured over time, as well. This essentially means that much of the subject matter has already been chosen for me.

My desire would be to measure the doubling of human knowledge by noting beneficial effects of new knowledge. If human knowledge in oncology has doubled in 5 years, the clearest proof for me would be the doubling of survival rates for cancer in the same period (although there would obviously be a time lag involved). That's for my next post.

But casualties to U.S. troops on the battlefield are measured, and historical information allows comparison. And, there is evidence of dramatic improvements in survival rates for casualties due to improvements in care. Note that not all improvements are due to specifically medical knowledge. Improvements in logistics (getting a wounded soldier to care, getting the right treatment to the soldier on the battlefield, improvements in body armor) contribute heavily to this improvement. However, this is all improved knowledge being applied to the medical treatment of injured soldiers. To me, this counts.

Here is the lead paragraph from the first result of a search on Google News for 'battlefield survival rates.' It comes from Heraldnet in an article titled "Trauma Care Boosts Battlefield Survival Rate."

"The survival rate for U.S. service members wounded in Iraq has reached 90 percent - higher than in any previous war and 10 points higher than in the 1991 Persian Gulf War.

The major reason, says the general in charge of Army medical training, is improved trauma care being delivered moments after injury by medics and a growing number of soldiers trained as combat lifesavers."

The U.S. Army Public Affairs site has an article about the improved survival rates as well, noting that "of the 1,077 soldiers injured in Operation Iraqi Freedom and Operation Enduring Freedom, only 18 have died of their wounds, a rate of only 1.6 percent. This is less than half the 3.68 death rate for wounded soldiers in Vietnam."

I would rather have this type of information about cancer survival rates. I don't think that battlefield survival rates necessarily should have a one-to-one correspondence with improvements in knowledge, training and logistics. But this is what is being written about. What it does show is that knowledge has improved dramatically--but it has improved incrementally and linearly, not geometrically, and the period is perhaps over a decade and not five years.

The final word shows more historical comparison, coming from Wounds1.com. "During World War II, approximately 30% (almost one-third) of soldiers wounded in combat died from their wounds. In the Korean War, the number dropped to about 24%, and held steady at that rate through the Vietnam and Persian Gulf Wars. In the current action in Iraq, the number has dropped dramatically to just under 14%, or one in seven." A 50% drop in death, but over a 60-year timespan. Truly great and wonderful news, unless you are assuming it will occur twice a decade.

Are there more people generating knowledge?

One proxy for measuring the creation of knowledge could be the number of people engaged in that activity. Not all knowledge is scientific (is it?), but governments tend to keep an eye on how many people are either practicing science or studying it. In the U.S. and the U.K., people are worried that the numbers are decreasing. Does the rest of the world make up for the shortcomings?

Well, someone has done some work on this: Via Google Answers we see... (and boy, is Google Answers great)

• The number of scientists in the United States doubled very quickly in the past. "the number of scientists increased from 150,000 to 2,685,000 between 1950 and 2001" but growth from 2001 projected to 2010 is much more modest, with only medical scientists having doubled in the previous decade and projected to grow at the substantial rate of... 26.5% in this decade we're in.
• According to the OECD Observer (January 2004), the number of researchers in the OECD rose 42%  from 1990-2000, which won't get the doubling job done. The article talked briefly about foreign researchers filling the gap...

So how about those Chinese scientists? Those Indian software geniuses?

• An article in Intellectual Property Watch says the number of patents filed in India has increased 50% in the past 5 years, without offering numbers. That's a big increase, but only half of what would be needed to double human knowledge, if we used patent filings as a proxy for new human knowledge (and found that percentage increases were similar worldwide). And if we are counting on India and China to fill a gap caused by U.S. and U.K. reductions in research and researchers, this would not support that thesis.
• In fact, patent applications to the World International Patent Office last year reached a record 145,300 in 2006, according to this article in Inc.com. But the article notes that this increase was only 6.4% year-on-year, and the United States accounted for one third, followed by Japan and Germany. This shows no evidence of 5-year doubling (you need an annual growth rate of 14.78% to double in five years), and no evidence that Asia will replace more traditional centers of knowledge production.

There Were Hints And Allegations

Remember we are testing three hypotheses here:

1. The actual store of human knowledge (not, repeat not, data) has doubled in the past 5 years

2. Access to knowledge has grown so quickly that it fulfills the same function as a doubling of human knowledge

3. We are disproving previously held theories at such a rate that it frees up resources to concentrate on useful lines of enquiry resulting in an effective doubling of knowledge.

The easiest to find positive indications for is number 2. As an example, 5 years ago:

• Wikipedia had 80,000 articles and 125,000 hits per day in August 2002. On January 1, 2007, it had over 6 million articles (in 251 languages!) and in Dec. 06 it averaged more than 5 million daily unique visits. Okay, that's more than double... but this only works for hypothesis number 2, as we have no indication on how much of the knowledge placed on Wikipedia during that time period existed before 2002.

It Should Be Lumpy, Right?

We wouldn't expect a uniform doubling of human knowledge within a 5-year timeframe, right? Even if this is a law as inexorable as Moore's (and that doesn't seem as inexorable as it once was, or am I reading the Dell ads wrong?), one would reasonably expect that human knowledge would advance a little in some areas and a lot in others. You with me?

Given what the world's attention has been focused on since 2002, here are some of the areas where I think human knowledge should have increased fastest:

• Epidemiology, thanks to SARS and Avian flu
• Biofuels
• Vaccines
• Proteome
• Oil extraction from rock and shale
• Mars
• Genetically Modified Organisms
• Fishery management
• Global warming
• Solar activity
• The Long Tail
• Stem cell cultivation
• Infectious diseases

I will treat this as a proxy list of subject areas to explore. I'm going to be digging around to see if we've seen dramatic advances in any or all of these. I'll probably dedicate a few posts to each of them. Got any suggestions?