Does Human Knowledge Double Every 5 Years?

Sobering News

The post below from yesterday, which details the stately, rather than exponential, increase in granted patents in the U.S., has really got me thinking, and I think we'd better report on the state of play here.

Let's detail those areas where we have seen adequate growth to double human knowledge every 5 years:

Update: We have since found other areas where the CAGR is sufficient to double human knowledge every five years:

Update 2: A modest review revealed errors in my calculations regarding oncology. Its position has changed (and not for the better).

Prions, where academic journal publications have increased at a CAGR of 25.57%, and patents have increased by 33.76%

Software, where academic journal publications have increased at a CAGR of 22.83% and patents have increased by 33.53%

Post-graduate enrollment in China, which has increased at a CAGR of 22.38% (although see below for more discouraging information about the growth of tertiary education worldwide)

(All percentages are CAGR (compound annual growth rate.)

1. Nanotechnology, with a blistering growth in publications of 97.16% per year. Patents, 76.15%.

2. Global Warming publications, with a of 16.85%

3. Total Chinese patent applications, at 25% per annum

4. Chinese computing patents, 58.29%

5. Chinese telecommunications patents, 30.26%

(was oncology)

7. Epidemiology patents, 22.76%

So far, that's it.

Now let's look at where we did not find anything like doubling every five years (remembering that we need a CAGR of 14.87% to double in five years time):

Update: We have now found other areas where human knowledge does not appear to be growing at a rate that will double in five years time:

Worldwide university enrollment, which has a CAGR of 4.85%

Alzheimer's disease, where academic publications have a CAGR of 9.65% and patents a CAGR of 11.26%

Programming, where academic publications have a CAGR of 12.55% and patents a CAGR of 12.83%

Chinese university enrollment, with a CAGR of 9.76%

1. U.S. patents granted, 5.21%

2. Epidemiology publications, 8.85%

3. Shale oil extraction, publications 5.59%, patents, 2.61%

4. Publications on Mars, 5.78% (see also here and here)

5. U.S. Book publishing, 5.14%

6. Indian university enrolment since 1947, 5.51%

7. Climate Models, 12.66% (Very close to 5-year doubling rate of 14.87%)

8. Improvements in Climate Models, 13.76% (Also very close)

9. Growth in publication of scientific and technical papers, 1995-1999

             OECD 0.1%

             U.S., -1.8%

             Canada, -2.2%

             UK, -0.1%

             Netherlands, -0.5%

10. Publications in astrophysics, 4% (since 1970)

11. Growth in percentage of cancer patients surviving 5 years after diagnosis, 0.7% (1960-2005)

12. Drop in percentage of soldiers dying from wounds received in battle, 1.54% (1941-2006)

13. Projected growth in number of scientists (U.S.), next ten years, 7.18%

14. Patent filings in India, last five years, 8.45%

15. Increase in patent filings, World International Patent Office, 1 year, 6.4%

6. Oncology pubications, 9.23% (revised)

Considering I cherry picked areas that I thought would be most likely to show robust growth, this is scary.

          

Human Knowledge and Mars

(This part is repeated from the previous post): NASA and the Smithsonian Institute maintain a database called the Astrophysics Data System. Harvard University is kind enough to host it here. Available to you and I is an abstracting service which will return brief descriptions of articles found by keyword searches. You see where I'm going here?

For a description of coverage and limitations of the database, click here. Then, let's have some fun.

A search for the word Mars in titles and abstracts performed on April 21 2007 returned 20,978 results.

(This is where the new stuff starts): Here we report on 5-year periods going back to 1900. We show annual percentage growth on the previous 5-year total. It isn't pretty--much more like real life than any smooth logarithmic curve. In 22 five-year periods, we show an annual percentage growth above 14% (shown in bold) in only 3 five-year periods(you need a 14.87% annual growth to double every five years). In 7 of the five-year periods, there is a decline in publications (inefficient post, wars, depressions, Mars being further away... lots of logical reasons why this might have occurred).

The serious question is, if our studies of Mars, which have benefited from major government funding from several large governments, and have also benefited from being topical, exciting, and having large sources of data available recently, do not show exponential growth, where would we expect to actually find a doubling of human knowledge every five years? If not here, where?

What I think it shows me is that when there is a lot of funding, real news, a research goal, and the availability of new and important data, we focus on the subject, do the research and improve knowledge. But for extended periods, things just sort of tick along, improving respectably in many years, but showing declines in others. What do you make of the following?

Click if you want to see the numbers.

Human Knowledge and Mars

NASA and the Smithsonian Institute maintain a database called the Astrophysics Data System. Harvard University is kind enough to host it here. Available to you and I is an abstracting service which will return brief descriptions of articles found by keyword searches. You see where I'm going here?

For a description of coverage and limitations of the database, click here. Then, let's have some fun.

A search for the word Mars in titles and abstracts performed on April 21 2007 returned 20,978 results. Let's pretend for the sake of argument that this represents all human knowledge ever gained about Mars. (This may be as close to the truth as we will ever get on this weblog when searching for information about fields of study--but it is obviously not true, so take it with a grain of salt.)

What is the CAGR (Compound Annual Growth Rate) in publications? Well, if we were really strict, we would measure from the date of the first document, which is... 1644. That would give us a CAGR of 2.32% and we would all get very depressed. But a more realistic starting point would be 1830, when regular publications started. This would remove only 10 documents from our total, and would yield a CAGR of 5.78%. That's much better, but far short of doubling every five years. (At that rate, human knowledge about Mars would double every 12.34 years.) Nonetheless, half of the documents in the database have been published since 1993. (Hey--that doubled in 14 years... is the rate of doubling slowing down?)

Just the metadata I got from the site is fascinating, and I'll be referring to it later, looking for short term bursts and lags (and wondering if Mars' distance from the Earth is one of the causes, for example). But here's some interesting information.

The most recent, published in April 2007, is titled "Subsurface Radar Sounding of the South Polar Layered Deposits of Mars and written by Plaut, Jeffrey J.; Picardi, Giovanni; Safaeinili, Ali; Ivanov, Anton B.; Milkovich, Sarah M.; Cicchetti, Andrea; Kofman, Wlodek; Mouginot, Jérémie; Farrell, William M.; Phillips, Roger J.; and 14 coauthors

Document #3001 is titled "Review of the trajectory and atmospheric structure reconstruction for Mars Pathfinder", written by Withers, Paul; Towner, Martin; Hathi, Brijen; Zarnecki, John

Document #6001 is titled "History and perspectives of scientific ballooning," by Frank Lefevre. Hmm. Well, document #6002 is "Stability and evolution of the climate system of Mars," written in 2001 by Stability and evolution of the climate system of Mars

Document #9001 is titled "Mobilization of dust on the Mars surface by the impact of small cosmic bodies," written in 1997 by Rybakov, V. A.; Nemtchinov, I. V.; Shuvalov, V. V.; Artemiev, V. I.; Medveduk, S. A.

Document #12001 is titled "Dry Carbonate Formation on Mars: A Plausible Sink for an Early Dense CO2 Atmosphere?", written in 1990 by Stephens, S. K.; Stevenson, D. J.

Document #15001 is titled "Stratigraphic relationships within Martian polar CAP deposits" written in 1982 by Howard, A. D.; Cutts, J. A.; Blasius, K. R.

Document #18001 is titled "The major Martian dust storms of 1971 and 1973" written in 1974 by Martin, L. J.

The very first document, written in 1644 by Grandami, Jacques, is titled "Nova demonstratio immobilitatis terrae petita EX virtvte magnetica."

If that's a little dense for you, the second, written in 1672 by Flamstead, John, is "An Extract of Mr. Flamstead's Letter Written to the Publisher from Derby Novemb. 16. 1672. Concerning the Appulses, by Him Calculated for A. 1673. of the Moon, and the Other Planets to Fixed Stars; Together with an Observation of the Planet Mars, Made by the Same"

Human Knowledge and Mars

I off-handedly chose Mars as a subject for investigation, basically because there have been several missions to the Red Planet in the past 5 years. Those missions were expensive, they were designed to gather knowledge about Mars... so if human knowledge is doubling every 5 years in any given sector, we might imagine that one of the sectors is Mars. Well, I actually have an ulterior motive, but that's for the end of this post.

The Exploration of Mars is beautifully detailed on Wikipedia. The first probe to actually get close to Mars was in 1962, the Soviet mission Beta Nu 1. It was followed by the US Mariner 4 in 1964. So we have been collecting data from space missions for 45 years. The 1996 Mars Global Surveyor effectively returned more data than all previous missions combined, and the data is available to the public.

The Mars Pathfinder landed on Mars on July 4, 1997, and a remote controlled rover called Sojourner decamped from the lander and went walkabout. Pathfinder returned 16,500 images from the lander and 550 images from the rover, as well as 15 chemical analyses of rock and soil and extensive data on winds and weather. This was followed by a spate of unsuccessful attempts to return to Mars.

In 2001 the run of bad luck ended when NASA's Mars Odyssey orbiter arrived. Its mission is to use spectrometers and imagers to hunt for evidence of past or present water and volcanic activity on Mars. In 2002, it was announced that the probe's gamma ray spectrometer and neutron spectrometer had detected large amounts of hydrogen, indicating that there are vast deposits of water ice in the upper three meters of Mars' soil within 60° latitude of the south pole.

NASA sent a pair of twin rovers toward the planet as part of the Mars Exploration Rover Mission. On 10 June 2003, NASA's MER-A (Spirit) Mars Exploration Rover was launched. It successfully landed in Gusev Crater (believed once to have been a crater lake) on 3 January 2004. It examined rock and soil for evidence of the area's history of water. On July 7, 2003, a second rover, MER-B (Opportunity) was launched. It landed on 24 January 2004 in Meridiani Planum (where there are large deposits of hematite, indicating the presence of past water) to carry out similar geological work.

Mars Reconnaissance Orbiter is a multipurpose spacecraft designed to conduct reconnaissance and exploration of Mars from orbit. The $720 million USD spacecraft was built by Lockheed Martin under the supervision of the Jet Propulsion Laboratory, launched August 12, 2005, and attained Martian orbit on March 10, 2006.

The MRO contains a host of scientific instruments such as the HiRISE camera, CRISM, and SHARAD. The HiRISE camera is used to analyze Martian landforms, whereas CRISM and SHARAD can detect water, ice, and minerals on and below the surface. (In case you haven't guessed, I'm just copying from Wikipedia here.)

Now, the Wikipedia article on Mars itself is quite long. It contains very little material that could have been written before the past 45 years--the planet was mapped in 1840, some basic astronomical information has been long known, but almost all of what Wikipedia thinks is important is recent, and most of it really, really recent. Of the 79 citations at the bottom of the article, 30 were written after 2004.

So I'm ready to admit that human knowledge about Mars has doubled every 5 years since 1962. I'll be searching for more evidence to back this up. But here's my ulterior motive: I think that new sectors, or sectors that have big money thrown at it, or sectors where there is a clear public good to be obtained from applied research, might well indeed experience exponential increases in knowledge. But are they sustained? Does knowledge gained transfer readily to similar sectors? (Do we, for example, know as much about Venus or Mercury?) If new knowledge gained in the past 5 years about Mars (and I include getting to Mars, data collection design, etc.) leads to similar increases in knowledge about other planets in our solar system, then I think we're on to something here.

It Should Be Lumpy, Right?

We wouldn't expect a uniform doubling of human knowledge within a 5-year timeframe, right? Even if this is a law as inexorable as Moore's (and that doesn't seem as inexorable as it once was, or am I reading the Dell ads wrong?), one would reasonably expect that human knowledge would advance a little in some areas and a lot in others. You with me?

Given what the world's attention has been focused on since 2002, here are some of the areas where I think human knowledge should have increased fastest:

• Epidemiology, thanks to SARS and Avian flu
• Biofuels
• Vaccines
• Proteome
• Oil extraction from rock and shale
• Mars
• Genetically Modified Organisms
• Fishery management
• Global warming
• Solar activity
• The Long Tail
• Stem cell cultivation
• Infectious diseases

I will treat this as a proxy list of subject areas to explore. I'm going to be digging around to see if we've seen dramatic advances in any or all of these. I'll probably dedicate a few posts to each of them. Got any suggestions?