Does Human Knowledge Double Every 5 Years?

Whew--We Find That Something Has Doubled

It would have been truly strange not to find some area where there is evidence of a doubling of activity. I actually think we'll find more as we go on. But the past two weeks have been somewhat discouraging, so I'm happy to report that we see clear evidence of a doubling in the growth rate of patent filings in China over the past 5 years, and a doubling of the total number of academic patent filings worldwide between 1995 and 2000.

An article titled 'Rapid Patenting Growth By Academic Institutes In The People's Republic Of China' by Simon Pratt, published in June 2005 shows a doubling of the growth rate (not the actual numbers) of patent filings by academic institutions in China, and a doubling of academic patent filings worldwide by universities between 1995 and 2000 (see chart below). Sadly, that growth rate has not been sustained since 2000.

It is not unalloyed good news--worldwide, academic patent filings are a response to governmental decisions to allow universities to retain, license and exploit intellectual property, so there may just be substitution here--the same patent might previously have been filed by an individual at the academic institution or someone who had access to the reseach. But I'm so happy to find something relevant that has doubled in five years that this seems like a minor quibble.

Here is the chart showing the number of academic patent filings worldwide since 1980. As with all the charts on this weblog, clicking on the chart will bring it up in a readable size.

More on Human Knowledge and Global Warming

In an article titled 'General Circulation Models of the Atmosphere,' Spencer Weart tell us that John von Neumann began advocating using computers to study the climate back in 1946, shortly after his computer ENIAC became operational. Getting grants from the Navy, the Air Force and the Weather Bureau, the study of climate began.

So 1946 was the starting point. For the next 25 years, the American Meteoroligical Society's  entire spectrum of journals (including the Journal of Applied Meteorology, the Bulletin of the American Meteorological Society, Journal of Atmospheric Sciences, Journal of Physical Oceanography, Monthly Weather Review and more...) published a total of 111 papers containing the phrase "climate models" somewhere in the text or title. The first one was titled 'The Unusual Weather and Circulation of the 1948-1949 Winter,' by William H. Klein. It was published in April 1949, 58 years ago. Two articles were published containing that phrase in 1950.

Let's pretend that each paper published with the phrase "climate models" is one unit of advancing human knowledge. If human knowledge doubles every five years, how many papers should be published in 2005? We know that the percentage growth should be 14.87%, and we know we should expect to see 2,048 articles in 2004. So... drum roll please... it's 881. Less than half of what we would expect. Lest you think it was just an off year, in 2005 it was 899 and in 2006 it actually dropped to 894.

I can think of a dozen possible explanations--a presidential administration unfriendly to the concept, waiting on the IPCC to publish findings in 2007... but the fact is, if we have doubled our knowledge of climate models in the past 5 years, we're not writing about it in America.

Is the rest of the world making up for America's shortfall? Well, not if you're depending on the International Journal of Meteorology, which returns 8 results for a search on climate models over a 25-year span. (But I think it's a search issue.) The International Journal for Climatology returns 37 results. I'm sure I'm missing important sources, but none are jumping up and biting me. The Royal Meteorological Society has 36 results. The UK government is considerably more supportive of research into global warming than the current American administration. I really thought there would be more.

Human Knowledge and Global Warming

A Google search for "improvements in climate models" returns 2.64 million results. Of those, Google Scholar claims 138,000 articles about the search term.

Of the first 20 results in Google Scholar, only one was written after the year 2000. Published in the Journal of Climate, it describes The Community Climate System Model, Version 2.  However, searching with the same term on the Journal of Climate's search engine returns 3,348 results.  Sorting in reverse chronological order, the first 10 were published this month, April 2007. In fact, I counted 279 entries published in the past year. Here are the annual results for fifth years historically:

2006: 273

2001: 203

1996: 162

1991: 54

1986: 49

1981: 24

1976: 14

1971: 3

1966: 0

Global warming is a popular field for researchers and government funding is there. Advances in computer hardware and software means that improvements in global climate models are possible, as well as desirable.

The fact that the last time entries doubled over a five-year span was between 1991 and 1996 is a bit disturbing. A new field of science with optimum research conditions that cannot sustain doubling of cutting edge knowledge after a mere 25 years cannot bode well for those who depend on doubling every five years.

Maybe the papers are four times as good...

Will Emerging Countries Double Human Knowledge For Us?

Maybe not--they may not even replace the shortfall in scientists and engineers in the OECD.

In December 2005, Vivek Wadhwa, a professor at Duke University, wrote an article that was published in Business Week. He was trying to allay fears of India and China as outsourcing monsters that would eat engineering jobs. Wadhwa said that Duke students were wondering if they should even enter the profession if the jobs would migrate to Asia. Wadhwa, Professor Gary Gereffi and some Duke students researched the topic. They published a report (large PDF) that may indeed lay some outsourcing fears to rest.

However, for those who are anticipating that a surge in Asian technical knowledge workers will step into the shoes of Western students who are abandoning technical fields of study, the report brings no comfort.

In China in 2004, 644,106 engineers graduated. However, only 351,537 were four-year degrees. Worse, due to differences in definitions, the latter figure includes motor mechanics and industrial technicians.

In India in 2004, a total of 215,000 engineering graduates were produced. However, only half (112,000) were four-year degrees--the remaining being 3 years.

In the US, a total of 70,000 engineers graduated with 4-year diplomas, a figure that rises to 222,325 if computer science graduates are included. Indeed, the number of science and engineering doctorate degrees awarded in the United States dropped by 7 percent from 1998 to 2001, according to a
separate National Science Foundation study made public this week. However, enrollment in science and engineering graduate programs rose in 1999 and 2000.

As I have not found historical data for China and India, I don't know if their output has not only doubled, but also taken over the load for the disappearing Western graduates--including the implied doubling that we are looking for. We do know that they have been pumping out engineers for quite a while, so their figures are unlikely to be doubling--just growing as fast as they can. So, this post isn't really finished. But it doesn't look as though the results will support the principal contention in Hypothesis 1, that human knowledge is doubling every five years.

More to follow.

A Marker is Put Down

Derek J. de Solla Price, writing in 1965 in Science, gives us a benchmark. In a paper entitled Networks of Scientific Papers, (PDF) exploring the links between citations and references, de Solla Price writes:

"We can calculate roughly that, since the body of world literature has been growing exponentially for a few centuries and probably will continue at its present rate of growth of about 7 percent per annum, there will be 7 new papers each year for every 100 previously published papers in a given field." A footnote refers to his 1963 book, Little Science, Big Science, published by Columbia University Press.

So, we return to the Rule of 72 website to find that a 7% growth rate would result in the doubling of scientific publications every 10.3 years.

Not good news, but that was in 1965. Is there more recent information?

Yes, and its worse news. The National Science Foundation published a chart showing growth in scientific and technical papers for selected countries. It's here--click on the image to see a readable version.

For the OECD, it shows very slow growth (1.5%) between 1987-1994, and no growth (0.1%) from 1995-1999.

This is really bad news. The higher growth rate doubles in 48 years, the lower growth rate in 720 years. I hope we've been making up for lost time since 1999...

One indication why this may have happened is reduced government funding for research and development. The graph above comes from a 2002 report called Science and Engineering Indicators. It writes, "The decline in the share of government funds for R&D is a key trend common to all major industrial nations and many other OECD countries. In the mid-1980s, these nations derived an average of 45 percent of their R&D funds from government sources; by 1998, this figure had fallen to less than one-third. The relative retrenchment reflects the broad growth of industrial R&D, reductions in defense R&D in some key nations, and broader economic and spending constraints on governments. As a consequence, government funding for industrial R&D performance also fell, averaging 23 percent in 1983 but only 10 percent in 1998 for OECD as a whole."

An Almost Eureka Moment

Bless you Tatyana Dorokhova, and bless the Astronomical Observatory of Odessa National University!

In your Abstract for your article, Exponential Growth of Scientific Information and Orientation in World Wide Web (published in the July 2-5, 2002 issue of Library and Information Services in Astronomy IV), you write,

"A swift growth in the number and length of publications in the  narrow, but advanced field of astronomy is shown with certain research examples."

However, although your title refers to exponential growth, your numbers do not. You quote H. Abt, an editor of the Astrophysical Journal, as writing "The average number of papers increased exponentially by nine percent before mid 1970's and by four percent per year thereafter."

Sadly, this look at postwar astronomy seems to show an explosion of knowledge after the war, aided by improvements in astronomical tools (radio telescopes, etc.), and then a return to a very respectable growth rate that would double human knowledge in this sector in 18 years. (Lest you think I actually did the math, I refer you to the Rule of 72 website, which has a doubling calculator.) Even the 9% growth rate before the mid-1970's would yield a doubling only once every 8 years.

Once again, we are frustrated in our attempts to show doubling of human knowledge every five years.

The Outcome Fallacy and the Limits of Desktop Research

Throughout this blog, I look for positive outcomes and improved results coming from increased human knowledge regarding big problems. I rather blithely say 'if we have doubled our knowledge regarding cancer, why aren't survival rates doubling?", and things like that.

But there is no reason to assume that this should be the case, and I know that. It could be that for a problem as difficult as cancer, a doubling of human knowledge will only produce a 10% improvement in survival rates. I don't know. I don't think anybody else does either. (For God's sake, if you do know, tell me.)

Intractable problems may not have a solution at all. Or we may need to multiply our knowledge 100 times before we can solve them.

But looking for a dramatic improvement in a problematic situation seems legitimate--and we have already found one specific instance where human knowledge has dramatically improved the outcomes of an intractable problem--battlefield survival rates.

So, fair warning. I'll continue to use language loosely and describe human knowledge and outcomes with only the metrics that Google proffers. On the other hand, if this blog ever starts gettng traffic, you will find me flexible about ways of improving both my language and my metrics.

In a Knowledge Economy, Is Global GDP a Proxy for Knowledge?

Let's assume for a moment that it might be. World GDP has been growing strongly of late--above trend, and in the face of challenges that formerly may have caused it to falter. But it has not grown at anything that would resemble a doubling every 5 years--in 2000, according to the World GDP was 32 trillion US dollars, and in 2005 it was 44.4 trillion. Great growth--not doubling.

You could point out the obvious--that knowledge is not spread evenly throughout the world. But the part of the world where knowledge might be presumed to be greatest grew at a slower rate than the part of the world where it might be presumed to be lesser. No doubt much of that growth is due to increased human knowledge. But more than a little was due to more workers.

Hmm. This would not be the first time that we see indications that the more dramatic effects of knowledge occur when existing knowledge is made available to new people. Making 30-year-old call centre technology available to India, disassembling factories in England and reconstructing them in China and Indonesia, this is knowledge transfer, not creation. And, to repeat my refrain, this is great, good and wonderful--if you don't have an agenda that demands exponential growth in new knowledge.

The search continues.

Why This is Important (I hope)

I don't fear death. And I won't fear death--unless and until there is an alternative.

A group of scientists and science popularizers are creating the expectation of an alternative to death. It's a pretty important concept, and a powerful hope to offer. The account I've read most recently is by Ray Kurzweil, author of The Singularity is Near. (If you're shopping from the UK, as I do, click here.) Kurzweil, as well as many of the other popularizers of the exponential growth of knowledge, argues that... well, something is growing so fast that within the lifetimes of many of us, machine generated knowledge will sort of take over from us. I'll be talking a lot about that 'something' in future posts.

For now, let me just say that if human knowledge is not doubling quickly enough to give exponential growth rates, a lot of what these people predict kinda falls on its face. Now, I want this future to come to pass--it holds too many potential benefits to ignore or wish away. (I do wish, however, that a more manageable version of this idea was available). But it all depends on the exponential growth of human knowledge.

Now, these people (and I'll be naming who I'm talking about later) talk a lot about exponential growth in information technology, etc., and I can well believe that the price of computing and storage is dropping exponentially. In fact, I know it has from personal experience. But having free memory (stored on a free weblog server, for example) and infinitely fast access to that memory does not mean a blessed thing if I don't have useful information to store and/or access.

So what's at stake is our future--gee, that sounds somewhat dramatic. But a lot of people who read these Singularity/Immortality/SENS/etc. protagonists need to know if this is science or science fiction. I'm one of those who need to know. That's why I'm doing this. Just so you know.

non-Conclusions on Medical Knowledge

After writing the previous three posts, I have some thoughts about human knowledge and medicine. I reserve the right to change my mind on any of this:

• Human knowledge about matters medical has increased dramatically, but does not double every 5 years
• Medical outcomes have increased human health and longevity, but these outcomes do not 'follow the money,' nor the research
• Managing newly acquired medical knowledge is now the key issue impeding further progress

The second hypothesis I am testing is whether or not the increasing availability of knowledge to a wider population can satisfy the criteria of doubling human knowledge. I believe that medicine may offer strong evidence for this.

The invention of the polio vaccine did not eradicate polio. It was the diffusion of the knowledge about the potential of the vaccine, first to doctors, then to mothers, that made the difference. Nobody has found much of a cure for lung cancer, but in areas where public policy has pushed awareness of the link between smoking and cancer (and social activism has become more militant), smaller percentages of the population continue to smoke, and this has had a beneficial effect.

The same is roughly true regarding the benefits of mammography, checking of the colon and prostate and pap smears. It is pushing the knowledge to the people that has made all the difference. Human life expectancy has risen in the developed countries to between 75 and 84 years of age. One hundred years ago, it was between 42 and 48. Almost all of the difference seems to me to be due to increases in human knowledge, but completely conditional on this knowledge being published and affecting the behaviour of the general public.