Does Human Knowledge Double Every 5 Years?

More on Human Knowledge and Global Warming

In an article titled 'General Circulation Models of the Atmosphere,' Spencer Weart tell us that John von Neumann began advocating using computers to study the climate back in 1946, shortly after his computer ENIAC became operational. Getting grants from the Navy, the Air Force and the Weather Bureau, the study of climate began.

So 1946 was the starting point. For the next 25 years, the American Meteoroligical Society's  entire spectrum of journals (including the Journal of Applied Meteorology, the Bulletin of the American Meteorological Society, Journal of Atmospheric Sciences, Journal of Physical Oceanography, Monthly Weather Review and more...) published a total of 111 papers containing the phrase "climate models" somewhere in the text or title. The first one was titled 'The Unusual Weather and Circulation of the 1948-1949 Winter,' by William H. Klein. It was published in April 1949, 58 years ago. Two articles were published containing that phrase in 1950.

Let's pretend that each paper published with the phrase "climate models" is one unit of advancing human knowledge. If human knowledge doubles every five years, how many papers should be published in 2005? We know that the percentage growth should be 14.87%, and we know we should expect to see 2,048 articles in 2004. So... drum roll please... it's 881. Less than half of what we would expect. Lest you think it was just an off year, in 2005 it was 899 and in 2006 it actually dropped to 894.

I can think of a dozen possible explanations--a presidential administration unfriendly to the concept, waiting on the IPCC to publish findings in 2007... but the fact is, if we have doubled our knowledge of climate models in the past 5 years, we're not writing about it in America.

Is the rest of the world making up for America's shortfall? Well, not if you're depending on the International Journal of Meteorology, which returns 8 results for a search on climate models over a 25-year span. (But I think it's a search issue.) The International Journal for Climatology returns 37 results. I'm sure I'm missing important sources, but none are jumping up and biting me. The Royal Meteorological Society has 36 results. The UK government is considerably more supportive of research into global warming than the current American administration. I really thought there would be more.

Human Knowledge and Global Warming

A Google search for "improvements in climate models" returns 2.64 million results. Of those, Google Scholar claims 138,000 articles about the search term.

Of the first 20 results in Google Scholar, only one was written after the year 2000. Published in the Journal of Climate, it describes The Community Climate System Model, Version 2.  However, searching with the same term on the Journal of Climate's search engine returns 3,348 results.  Sorting in reverse chronological order, the first 10 were published this month, April 2007. In fact, I counted 279 entries published in the past year. Here are the annual results for fifth years historically:

2006: 273

2001: 203

1996: 162

1991: 54

1986: 49

1981: 24

1976: 14

1971: 3

1966: 0

Global warming is a popular field for researchers and government funding is there. Advances in computer hardware and software means that improvements in global climate models are possible, as well as desirable.

The fact that the last time entries doubled over a five-year span was between 1991 and 1996 is a bit disturbing. A new field of science with optimum research conditions that cannot sustain doubling of cutting edge knowledge after a mere 25 years cannot bode well for those who depend on doubling every five years.

Maybe the papers are four times as good...

Will Emerging Countries Double Human Knowledge For Us?

Maybe not--they may not even replace the shortfall in scientists and engineers in the OECD.

In December 2005, Vivek Wadhwa, a professor at Duke University, wrote an article that was published in Business Week. He was trying to allay fears of India and China as outsourcing monsters that would eat engineering jobs. Wadhwa said that Duke students were wondering if they should even enter the profession if the jobs would migrate to Asia. Wadhwa, Professor Gary Gereffi and some Duke students researched the topic. They published a report (large PDF) that may indeed lay some outsourcing fears to rest.

However, for those who are anticipating that a surge in Asian technical knowledge workers will step into the shoes of Western students who are abandoning technical fields of study, the report brings no comfort.

In China in 2004, 644,106 engineers graduated. However, only 351,537 were four-year degrees. Worse, due to differences in definitions, the latter figure includes motor mechanics and industrial technicians.

In India in 2004, a total of 215,000 engineering graduates were produced. However, only half (112,000) were four-year degrees--the remaining being 3 years.

In the US, a total of 70,000 engineers graduated with 4-year diplomas, a figure that rises to 222,325 if computer science graduates are included. Indeed, the number of science and engineering doctorate degrees awarded in the United States dropped by 7 percent from 1998 to 2001, according to a
separate National Science Foundation study made public this week. However, enrollment in science and engineering graduate programs rose in 1999 and 2000.

As I have not found historical data for China and India, I don't know if their output has not only doubled, but also taken over the load for the disappearing Western graduates--including the implied doubling that we are looking for. We do know that they have been pumping out engineers for quite a while, so their figures are unlikely to be doubling--just growing as fast as they can. So, this post isn't really finished. But it doesn't look as though the results will support the principal contention in Hypothesis 1, that human knowledge is doubling every five years.

More to follow.

A Marker is Put Down

Derek J. de Solla Price, writing in 1965 in Science, gives us a benchmark. In a paper entitled Networks of Scientific Papers, (PDF) exploring the links between citations and references, de Solla Price writes:

"We can calculate roughly that, since the body of world literature has been growing exponentially for a few centuries and probably will continue at its present rate of growth of about 7 percent per annum, there will be 7 new papers each year for every 100 previously published papers in a given field." A footnote refers to his 1963 book, Little Science, Big Science, published by Columbia University Press.

So, we return to the Rule of 72 website to find that a 7% growth rate would result in the doubling of scientific publications every 10.3 years.

Not good news, but that was in 1965. Is there more recent information?

Yes, and its worse news. The National Science Foundation published a chart showing growth in scientific and technical papers for selected countries. It's here--click on the image to see a readable version.

For the OECD, it shows very slow growth (1.5%) between 1987-1994, and no growth (0.1%) from 1995-1999.

This is really bad news. The higher growth rate doubles in 48 years, the lower growth rate in 720 years. I hope we've been making up for lost time since 1999...

One indication why this may have happened is reduced government funding for research and development. The graph above comes from a 2002 report called Science and Engineering Indicators. It writes, "The decline in the share of government funds for R&D is a key trend common to all major industrial nations and many other OECD countries. In the mid-1980s, these nations derived an average of 45 percent of their R&D funds from government sources; by 1998, this figure had fallen to less than one-third. The relative retrenchment reflects the broad growth of industrial R&D, reductions in defense R&D in some key nations, and broader economic and spending constraints on governments. As a consequence, government funding for industrial R&D performance also fell, averaging 23 percent in 1983 but only 10 percent in 1998 for OECD as a whole."

An Almost Eureka Moment

Bless you Tatyana Dorokhova, and bless the Astronomical Observatory of Odessa National University!

In your Abstract for your article, Exponential Growth of Scientific Information and Orientation in World Wide Web (published in the July 2-5, 2002 issue of Library and Information Services in Astronomy IV), you write,

"A swift growth in the number and length of publications in the  narrow, but advanced field of astronomy is shown with certain research examples."

However, although your title refers to exponential growth, your numbers do not. You quote H. Abt, an editor of the Astrophysical Journal, as writing "The average number of papers increased exponentially by nine percent before mid 1970's and by four percent per year thereafter."

Sadly, this look at postwar astronomy seems to show an explosion of knowledge after the war, aided by improvements in astronomical tools (radio telescopes, etc.), and then a return to a very respectable growth rate that would double human knowledge in this sector in 18 years. (Lest you think I actually did the math, I refer you to the Rule of 72 website, which has a doubling calculator.) Even the 9% growth rate before the mid-1970's would yield a doubling only once every 8 years.

Once again, we are frustrated in our attempts to show doubling of human knowledge every five years.

In a Knowledge Economy, Is Global GDP a Proxy for Knowledge?

Let's assume for a moment that it might be. World GDP has been growing strongly of late--above trend, and in the face of challenges that formerly may have caused it to falter. But it has not grown at anything that would resemble a doubling every 5 years--in 2000, according to the World GDP was 32 trillion US dollars, and in 2005 it was 44.4 trillion. Great growth--not doubling.

You could point out the obvious--that knowledge is not spread evenly throughout the world. But the part of the world where knowledge might be presumed to be greatest grew at a slower rate than the part of the world where it might be presumed to be lesser. No doubt much of that growth is due to increased human knowledge. But more than a little was due to more workers.

Hmm. This would not be the first time that we see indications that the more dramatic effects of knowledge occur when existing knowledge is made available to new people. Making 30-year-old call centre technology available to India, disassembling factories in England and reconstructing them in China and Indonesia, this is knowledge transfer, not creation. And, to repeat my refrain, this is great, good and wonderful--if you don't have an agenda that demands exponential growth in new knowledge.

The search continues.

Why This is Important (I hope)

I don't fear death. And I won't fear death--unless and until there is an alternative.

A group of scientists and science popularizers are creating the expectation of an alternative to death. It's a pretty important concept, and a powerful hope to offer. The account I've read most recently is by Ray Kurzweil, author of The Singularity is Near. (If you're shopping from the UK, as I do, click here.) Kurzweil, as well as many of the other popularizers of the exponential growth of knowledge, argues that... well, something is growing so fast that within the lifetimes of many of us, machine generated knowledge will sort of take over from us. I'll be talking a lot about that 'something' in future posts.

For now, let me just say that if human knowledge is not doubling quickly enough to give exponential growth rates, a lot of what these people predict kinda falls on its face. Now, I want this future to come to pass--it holds too many potential benefits to ignore or wish away. (I do wish, however, that a more manageable version of this idea was available). But it all depends on the exponential growth of human knowledge.

Now, these people (and I'll be naming who I'm talking about later) talk a lot about exponential growth in information technology, etc., and I can well believe that the price of computing and storage is dropping exponentially. In fact, I know it has from personal experience. But having free memory (stored on a free weblog server, for example) and infinitely fast access to that memory does not mean a blessed thing if I don't have useful information to store and/or access.

So what's at stake is our future--gee, that sounds somewhat dramatic. But a lot of people who read these Singularity/Immortality/SENS/etc. protagonists need to know if this is science or science fiction. I'm one of those who need to know. That's why I'm doing this. Just so you know.

non-Conclusions on Medical Knowledge

After writing the previous three posts, I have some thoughts about human knowledge and medicine. I reserve the right to change my mind on any of this:

• Human knowledge about matters medical has increased dramatically, but does not double every 5 years
• Medical outcomes have increased human health and longevity, but these outcomes do not 'follow the money,' nor the research
• Managing newly acquired medical knowledge is now the key issue impeding further progress

The second hypothesis I am testing is whether or not the increasing availability of knowledge to a wider population can satisfy the criteria of doubling human knowledge. I believe that medicine may offer strong evidence for this.

The invention of the polio vaccine did not eradicate polio. It was the diffusion of the knowledge about the potential of the vaccine, first to doctors, then to mothers, that made the difference. Nobody has found much of a cure for lung cancer, but in areas where public policy has pushed awareness of the link between smoking and cancer (and social activism has become more militant), smaller percentages of the population continue to smoke, and this has had a beneficial effect.

The same is roughly true regarding the benefits of mammography, checking of the colon and prostate and pap smears. It is pushing the knowledge to the people that has made all the difference. Human life expectancy has risen in the developed countries to between 75 and 84 years of age. One hundred years ago, it was between 42 and 48. Almost all of the difference seems to me to be due to increases in human knowledge, but completely conditional on this knowledge being published and affecting the behaviour of the general public.

The War on Cancer

There are many who say that we are losing the war on cancer. This article (it is from Fortune, reproduced in PDF format on the Aperio blog), offers depressing information. It says (in part)

• The age-adjusted percentage of Americans dying from cancer is the same now as it was in the 1970's... and the 1950's
• That 33 years ago, half of all cancer patients survived 5 years or more following diagnosis. That figure has climbed, but to only 63%
• Most of the progress that has been made has been due to early identification of the disease
• American spending (all public and identifiable private sources) on cancer research amounts to $200 billion since 1971
• The cancer research community has published 1.56 million papers just on signaling pathways and its related genes

And yet... for all the article's accounts of false starts, corporate greed and bureaucratic fumbling, a great body of knowledge has been accumulated. A lot of wrong paths have been followed--but if they are identified and no more time and effort is wasted on them, it means something. Those with many years of experience in the field are starting to talk about drug cocktail approaches that target both the tumor and the environment it lives in. This is after watching AIDS treatment make spectacular rapid progress while cancer treatment seemed stalled. There is talk about imitating strategies that were successful in the rapid reduction of heart attacks and strokes, as well. And again, a lot of basic research has been done.

Knowledge has been created. Outcomes have not matched this creation. Politics and corporate policy have interfered. But it looks... possible... that when the politics and corporate policy are finally removed, that progress can jump and outcomes catch up. One hopes.

Cancer Survival Rates

Again, a doubling of human knowledge should result in different outcomes in areas where that knowledge is applied. A lot of money is spent in improving our knowledge of certain cancers. If we double our knowledge, treatment of cancer should have a beneficial effect--improved survival rates being one effect we can measure.

A Google news search on cancer survival rates shows evidence of this. From Canada, the National Post reports that "the death rate for breast cancer for Canadian women has fallen 25% since 1986, when organized breast screening programs began being rolled out across the country." Apparently, this reduction is down to human knowledge from the early 1980's (beneficial effects of mammography) being dispersed through the wider population. Again, to me this counts. This is good news, great news. But again, if we keep a 5-year clock ticking, this news will disappoint, as it shows nothing remotely indicating a doubling of human knowledge within that timeframe.

Similarly, MedicineNet says, "Statistics (released in 1997) show that cancer patients are living longer and even "beating" the disease. Information released at an AMA sponsored conference for science writers, showed that the death rate from the dreaded disease has decreased by 3 percent in the last few years. In the  1940's only one patient in four survived on the average. By the 1960's, that figure was up to one in three, and now has reached 50% survival." So we see a doubling of the beneficial effect, but over 60 years, not five.

More encouragingly, St. Jude's Research Hospital shows 5-year survival rates for some cancers since 1962, and some of them might just qualify as 10-year doubling (pause to experiment on how to present this information... Okay, got it--click on the image to enlarge to readable size).

Again, a miracle, but it doesn't fit a 5-year window--it's 10 years for the best performing cancers.