Does Human Knowledge Double Every 5 Years?

Sobering News

The post below from yesterday, which details the stately, rather than exponential, increase in granted patents in the U.S., has really got me thinking, and I think we'd better report on the state of play here.

Let's detail those areas where we have seen adequate growth to double human knowledge every 5 years:

Update: We have since found other areas where the CAGR is sufficient to double human knowledge every five years:

Update 2: A modest review revealed errors in my calculations regarding oncology. Its position has changed (and not for the better).

Prions, where academic journal publications have increased at a CAGR of 25.57%, and patents have increased by 33.76%

Software, where academic journal publications have increased at a CAGR of 22.83% and patents have increased by 33.53%

Post-graduate enrollment in China, which has increased at a CAGR of 22.38% (although see below for more discouraging information about the growth of tertiary education worldwide)

(All percentages are CAGR (compound annual growth rate.)

1. Nanotechnology, with a blistering growth in publications of 97.16% per year. Patents, 76.15%.

2. Global Warming publications, with a of 16.85%

3. Total Chinese patent applications, at 25% per annum

4. Chinese computing patents, 58.29%

5. Chinese telecommunications patents, 30.26%

(was oncology)

7. Epidemiology patents, 22.76%

So far, that's it.

Now let's look at where we did not find anything like doubling every five years (remembering that we need a CAGR of 14.87% to double in five years time):

Update: We have now found other areas where human knowledge does not appear to be growing at a rate that will double in five years time:

Worldwide university enrollment, which has a CAGR of 4.85%

Alzheimer's disease, where academic publications have a CAGR of 9.65% and patents a CAGR of 11.26%

Programming, where academic publications have a CAGR of 12.55% and patents a CAGR of 12.83%

Chinese university enrollment, with a CAGR of 9.76%

1. U.S. patents granted, 5.21%

2. Epidemiology publications, 8.85%

3. Shale oil extraction, publications 5.59%, patents, 2.61%

4. Publications on Mars, 5.78% (see also here and here)

5. U.S. Book publishing, 5.14%

6. Indian university enrolment since 1947, 5.51%

7. Climate Models, 12.66% (Very close to 5-year doubling rate of 14.87%)

8. Improvements in Climate Models, 13.76% (Also very close)

9. Growth in publication of scientific and technical papers, 1995-1999

             OECD 0.1%

             U.S., -1.8%

             Canada, -2.2%

             UK, -0.1%

             Netherlands, -0.5%

10. Publications in astrophysics, 4% (since 1970)

11. Growth in percentage of cancer patients surviving 5 years after diagnosis, 0.7% (1960-2005)

12. Drop in percentage of soldiers dying from wounds received in battle, 1.54% (1941-2006)

13. Projected growth in number of scientists (U.S.), next ten years, 7.18%

14. Patent filings in India, last five years, 8.45%

15. Increase in patent filings, World International Patent Office, 1 year, 6.4%

6. Oncology pubications, 9.23% (revised)

Considering I cherry picked areas that I thought would be most likely to show robust growth, this is scary.

          

How Would We Define Victory?

Well, time to define a few things here. We are trying to discover if human knowledge doubles every 5 years. To do this, we are testing three hypotheses:

1. We learn so much new information that the actual stock of knowledge is double what it was 5 years ago. We are testing this by seeing if scientific publications and patents are doubling every 5 years. We also look and see if there are twice as many scientists and researchers as 5 years ago.
2. That new and old human knowledge is accessible and usable to twice as many people as it was 5 years ago. We're looking at things like the size and reach of websites like Wikipedia, and also referring to our search for more creators and users of information.
3. We are disproving incorrect hypotheses at such a rate that we are doubling human knowledge by unlearning untruths, which corrects mistakes and frees up time for productive learning.

But this is a bit simplistic.  If I know one thing and learn another, have I doubled human knowledge or just what I have learned? Is a learned fact human knowledge before it is published?

If I know one thing and, without learning another, communicate that one thing to another human being, does that double human knowledge because two of us now know it? Or is that just repetition?

If I discover an hypothesis is not correct, does it make any difference if others continue to hold the old and incorrect belief?

And then we must make things difficult by defining knowledge. Many of you will have seen the pyramid that has data at the bottom, information on top of it, then knowledge and then wisdom. The implication is that data is a fact revealed, information is the fact in context, knowledge is is the fact absorbed and assimilated, and wisdom is right action based on knowledge as experienced. But is that an accurate or adequate definition of knowledge? Is it sufficient?

Wikipedia has a definition--it's a bit long, but the start is promising. Of course the start is a quote from Oxford's English Dictionary. "Knowledge is defined (Oxford English Dictionary) variously as (i) facts, information, and skills acquired by a person through experience or education; the theoretical or practical understanding of a subject, (ii) what is known in a particular field or in total; facts and information or (iii) awareness or familiarity gained by experience of a fact or situation." Yeah, that's it.

So does learning that the Earth is more than 6,000 years old mean anything if millions don't believe it? If creationists make learned scientists spend time in court patiently explaining evolution instead of working in the laboratory--their true 'field of the Lord,' have we learned anything? Does knowledge have to be unanimously held, or unanimously available?

Boy, where's a philosopher when you need one?

Back to Work

I'll be writing a lot more about the Singularity later. Let's go back to the basic building blocks that will educate my later writing.

We return to the numbers of people in higher education. If this number is growing exponentially, there might be reason to think human knowledge will too, in future. Specifically, in India, we find "India's provision for higher education rose from a meagre 200,000 at the time of Independence to an astronomical 5,000,000 college and university places by 1998." Those words come from Gajaraj Dhanarajan, President and CEO, The Commonwealth of Learning, Vancouver, Canada. He was speaking in August of 2000 at the 11th General Conference--Universities as the Gateway to the Future.

Let's see--India achieved independence in 1947 (right? Might be off a year--Nope. Google says August 15, 1947). If Indian university students had doubled in number every five years since then, starting from a base of 200,000, they should now have 516 million university students. So, if they have started growing exponentially, it is recently. (How many university students should India have? Using figures from Dhanarajan's speech, to have the same rate of university enrollment as Canada {5%}, for example, India should have had 50 million university students in 2000. Room for improvement here).

This is actually fairly important. The Compound Annual Growth Rate (CAGR) for the Indian university population is 5.51%. The CAGR needed to double in five years is 14.87%. A CAGR of 5.51% means the population doubles every 13 years, instead of five. We won't get where we need to be at this rate.

If you've been reading from the bottom up, you'll be able to join in the refrain--"Great growth. Not doubling."

Will Emerging Countries Double Human Knowledge For Us?

Maybe not--they may not even replace the shortfall in scientists and engineers in the OECD.

In December 2005, Vivek Wadhwa, a professor at Duke University, wrote an article that was published in Business Week. He was trying to allay fears of India and China as outsourcing monsters that would eat engineering jobs. Wadhwa said that Duke students were wondering if they should even enter the profession if the jobs would migrate to Asia. Wadhwa, Professor Gary Gereffi and some Duke students researched the topic. They published a report (large PDF) that may indeed lay some outsourcing fears to rest.

However, for those who are anticipating that a surge in Asian technical knowledge workers will step into the shoes of Western students who are abandoning technical fields of study, the report brings no comfort.

In China in 2004, 644,106 engineers graduated. However, only 351,537 were four-year degrees. Worse, due to differences in definitions, the latter figure includes motor mechanics and industrial technicians.

In India in 2004, a total of 215,000 engineering graduates were produced. However, only half (112,000) were four-year degrees--the remaining being 3 years.

In the US, a total of 70,000 engineers graduated with 4-year diplomas, a figure that rises to 222,325 if computer science graduates are included. Indeed, the number of science and engineering doctorate degrees awarded in the United States dropped by 7 percent from 1998 to 2001, according to a
separate National Science Foundation study made public this week. However, enrollment in science and engineering graduate programs rose in 1999 and 2000.

As I have not found historical data for China and India, I don't know if their output has not only doubled, but also taken over the load for the disappearing Western graduates--including the implied doubling that we are looking for. We do know that they have been pumping out engineers for quite a while, so their figures are unlikely to be doubling--just growing as fast as they can. So, this post isn't really finished. But it doesn't look as though the results will support the principal contention in Hypothesis 1, that human knowledge is doubling every five years.

More to follow.

Are there more people generating knowledge?

One proxy for measuring the creation of knowledge could be the number of people engaged in that activity. Not all knowledge is scientific (is it?), but governments tend to keep an eye on how many people are either practicing science or studying it. In the U.S. and the U.K., people are worried that the numbers are decreasing. Does the rest of the world make up for the shortcomings?

Well, someone has done some work on this: Via Google Answers we see... (and boy, is Google Answers great)

• The number of scientists in the United States doubled very quickly in the past. "the number of scientists increased from 150,000 to 2,685,000 between 1950 and 2001" but growth from 2001 projected to 2010 is much more modest, with only medical scientists having doubled in the previous decade and projected to grow at the substantial rate of... 26.5% in this decade we're in.
• According to the OECD Observer (January 2004), the number of researchers in the OECD rose 42%  from 1990-2000, which won't get the doubling job done. The article talked briefly about foreign researchers filling the gap...

So how about those Chinese scientists? Those Indian software geniuses?

• An article in Intellectual Property Watch says the number of patents filed in India has increased 50% in the past 5 years, without offering numbers. That's a big increase, but only half of what would be needed to double human knowledge, if we used patent filings as a proxy for new human knowledge (and found that percentage increases were similar worldwide). And if we are counting on India and China to fill a gap caused by U.S. and U.K. reductions in research and researchers, this would not support that thesis.
• In fact, patent applications to the World International Patent Office last year reached a record 145,300 in 2006, according to this article in Inc.com. But the article notes that this increase was only 6.4% year-on-year, and the United States accounted for one third, followed by Japan and Germany. This shows no evidence of 5-year doubling (you need an annual growth rate of 14.78% to double in five years), and no evidence that Asia will replace more traditional centers of knowledge production.

It Should Be Lumpy, Right?

We wouldn't expect a uniform doubling of human knowledge within a 5-year timeframe, right? Even if this is a law as inexorable as Moore's (and that doesn't seem as inexorable as it once was, or am I reading the Dell ads wrong?), one would reasonably expect that human knowledge would advance a little in some areas and a lot in others. You with me?

Given what the world's attention has been focused on since 2002, here are some of the areas where I think human knowledge should have increased fastest:

• Epidemiology, thanks to SARS and Avian flu
• Biofuels
• Vaccines
• Proteome
• Oil extraction from rock and shale
• Mars
• Genetically Modified Organisms
• Fishery management
• Global warming
• Solar activity
• The Long Tail
• Stem cell cultivation
• Infectious diseases

I will treat this as a proxy list of subject areas to explore. I'm going to be digging around to see if we've seen dramatic advances in any or all of these. I'll probably dedicate a few posts to each of them. Got any suggestions?