Does Human Knowledge Double Every 5 Years?

The Pot of (Black) Gold

Oil Shale Extraction could bring access to 1 trillion barrels of oil in the United States. Canada has even more. If some biblical figure could strike his staff against the stone and bring forth oil, he (or she) would be a very rich biblical figure, indeed.

But it's a bit tougher than that. An interesting article about the state of oil today notes that "companies developing tar sand production in Alberta announced that their costs of production were rising substantially."

Sounds like a job for science and technology... especially as biblical figures are a bit short on the ground these days. With rumors of oil production about to decline and continued resistance to nuclear power, someone who can bring this large scale energy source to our gas tanks will be heavily rewarded.

So what is happening in this subsector of a subsector? There are academic institutions that focus on engineering and others that focus on energy. There are patents to be filed, alliances to be made, deals to be done and advances to be discovered. Again, like nanotechnology and global warming, this is one area where we should expect to find a doubling of human knowledge every five years--at least.

Better yet, from our point of view, we've been extracting oil from shale since 1847 (those canny Scots at it again), so there should be a time series of publications for us to study--almost as long as our study of studying Mars.

We did find one, at Scirus again. It goes back to 1920, but only gives results for the decade up til 1950.

Taking an average of 4 journal articles for 1920 and 115 patents for the same year, we get CAGR (Compound Annual Growth Rate) of 5.59% for journal articles, and 2.61% for patents. As shown below, you can easily tell when oil prices rose.

But comparing 2002-2006 with the five years preceding, we can quite clearly see the effects of commerce on knowledge seeking. Journal articles during this period rose by 13.94% for the entire 5-year period. But patents--they rose by 99.19%, doubling within 5 years.

We see similar effects when measuring output in the 70's. Journal articles grew more strongly--at 245.86% for the five year periods. Patents grew strongly, but only rose from 1,098 to 1,730 or 57.56%. In fact, it makes me wonder if the oil shocks of the 70's drove academic interest immediately, which led to a knowledge base being available for patent seekers when prices rose again after 2001.

The story of oil shale and human knowledge looks much more like punctuated equilibrium than the steady advance of human knowledge, and what punctures the equilibrium is economics.

Results below...

Back to Global Warming

And back to Scirus, the compendium of scientific publication. Okay, it's a search engine returning titles of journals, patents, etc.

What are we to make of the fact that searching for global warming on Scirus returns 100,000 more results than a similar search for nanotechnology? It's been studied for 20 years longer than nanotechnology, so global warming should reflect that... But there are 8,000 fewer academic articles in the database (28,000 for nanotech vs. 20,584 for global warming). What does that mean?

We found 5,184 patents on the Scirus database for nanotechnology. For global warming... 9,419. Scientists and technologists have been studying global warming for more than twice as long, so that may mean nothing. But, let's look at the statistics and see what they tell us.

As shown below, playing by the same rules as we did for nanotechnology and the study of Mars, we hereby proclaim a base year of 1975, as it saw 18 journal articles published and one patent filed. That gives us 31 years to measure growth. This indeed gives us a CAGR (Compound Annual Growth Rate) of 16.85%, which in fact does show doubling every five years! Hooray, we have another winner!

The CAGR for patents is higher yet--26.08%. (But again, it seems to have leveled off in 2002.) We saw below that growth was disappointing in the subsector of climate modelling, but it's being more than made up for elsewhere.

But, given the potential gravity of the consequences of global warming, am I the only one to look at the growth rates of human knowledge in this field, compare them to the growth rates for nanotechnology, and feel slightly disappointed?

The data is below. Click if you want to see the numbers.

But This Isn't Just About The Singularity, Dammit!

The Singularitarians need an exponential growth of human knowledge to enable their theory of short term artificial intelligence.

But I submit they are not the only ones who are changing their behaviour and expectations because of this general idea floating around. The idea that human knowledge doubles every five years can become an excuse for inaction.

If people think that human knowledge doubles every five years, they may think that improved science and technology will provide a 'deus ex machina' last minute cavalry charge over the hill solution to something, meaning they don't have to change their current behaviour.

They might continue smoking, thinking that surely, by the time they're 69 (the average age of diagnosis for cancer), we will have found a silver-bullet solution for all cancers.

Their organisations don't modify damaging behaviour, thinking that we will find a way to turn pollution into ice cream and champagne, or replace depleted fisheries with genetically modified de-boned cod that jump into our nets with a smile.

I think this attitude, to the extent that it exists, is quite dangerous. If my first pass through publicly available data suggests anything (and even when I finish it will need to be approached academically), it suggests that human knowledge does not double every five years.

What we do see is that specific sectors, such as nanotechnology or proteomics, grow even faster than that for short periods of time:

• When the sector is new
• When the sector is sexy
• When government or private industry sees a short term gain resulting from it
• When intermediate goals are visible and attainable

But exponential growth then reverts to a mean, or returns to an average growth that is far slower. That growth rate varies according to the availability of inputs, such as researchers, funding and above all the availability of findings from outside the given sector--the multi-disciplinary rescue. This has rescued Moore's Law more than once, and it is applicable across fields.

Not every problem deserves the resources of a Manhattan Project, which is the prime example of how throwing money and scientists at an issue can produce results. Some problems can be solved without a Manhattan Project. But I fear that problems like lung cancer and artificial intelligence do need resources at that scale...

...and I fear even more that a general, sloppy impression that human knowledge doubles every five years without any real effort or a need to get behind the knowledge machine and push is actually harming the cause of scientific and technological progress. It relieves us of personal responsibility. Oh, the new horde of Chinese researchers will solve it, so we don't have to.

That is pernicious.

What Is The Singularity?

Okay. Here is the definition provided by the Singularity Institute for Artificial Intelligence (SIAI):

"The Singularity is the technological creation of smarter-than-human intelligence. There are several technologies that are often mentioned as heading in this direction. The most commonly mentioned is probably Artificial Intelligence, but there are others: direct brain-computer interfaces, biological augmentation of the brain, genetic engineering, ultra-high-resolution scans of the brain followed by computer emulation. Some of these technologies seem likely to arrive much earlier than the others, but there are nonetheless several independent technologies all heading in the direction of the Singularity – several different technologies which, if they reached a threshold level of sophistication, would enable the creation of smarter-than-human intelligence."

The Institute is described on Wikipedia here. I emailed the founder, Eliezer Yudkowsky, to get his thoughts on what I am doing here. He hasn't responded as yet.

For 'Singularitarians', the Singularity is a big transformative concept, one of those spectacular theories that will change human life as we know it. When we create intelligence greater than our own, we will solve problems that we cannot solve today. Furthermore, many of the technologies (and especially nanotechnology) that will be used to create artificial intelligence will be used by the artificial intelligence to help us with healthcare, energy (PDF), space travel and more. God help us if we create an artificial intelligence that wants to sit on the couch and watch the soaps.

Those who fear artificial intelligence, nanotechnology (and perhaps big transformativc concepts), manage to create a vision of some rough beast slouching towards Bethlehem waiting to be born. From fears of grey goo (nanotechnology run wild) to the destruction of human ambition, AI is a threat--perhaps the biggest outsourcing threat of all, especially if it is embodied in robots.

Nanotechnology and Human Knowledge

There's a search engine called Scirus for scientific information. A search performed on Sunday, 22 April 2007 with the search term 'nanotechnology' returned 637,500 results. Fortunately, we can winnow that down to 28,846 journal results and sort them by date. (We also found reference to 5,148 patents) So, here we go again... 23,489 of those journal articles were published after 2002 ended. That's 81% of the total. Similarly, 4,494 patents were filed since the end of 2002, 87% of the total.

Using 1987 as a base year, we find CAGR in journal publications of 42.03%, which would double in 1.98 years. For patents, we see a CAGR of 44.91%, which would double every 1.87 years.

And this is why Ray Kurzweil and the rest of the Singularity gang get excited. Remember we need annual growth of 14.87% to double in five years. Nanotechnology has got it in spades.

Last year's growth was 37.87%. Maybe they will take over the world... but I will explain later why I think growth in nanotechnology is a necessary, but not sufficent pre-requisite for the Singularity.

Note that the advanced search by year  yielded 28,863 journal results (but it found all 5,148 patents )(one reason these types of databases drive academic researchers crazy--I hear Google Scholar is worse, and I'm afraid I'm going to find out soon...)

Click if you want to see the numbers

Chinese Papers in Nanotechnology

Well, I was going to wait before starting on nanotechnology, but I don't have a convenient place to park this information, so you get it in toto:

From PhyiscsToday.org: "A paper by Ping Zhou and Loet Leydesdorff (PDF) suggests that China is rapidly developing research expertise in nanotechnology. Six years ago, US-based scientists published 50% of the papers in the journal Nanotechnology and China accounted for zero. Today 25% of papers in the journal are from US-based scientists and 15% from China. Zhou and Leydesdorff report that between 1999 and 2005, for 85 journals in the fields of chemistry, physics and material sciences that are relevant to nanotechnology, the number of papers published by China rose exponentially to 8.34%, while papers published by other countries either remained steady or declined. China's overall world share in science journals is 6.52%, suggesting that China is aiming to be at the forefront of research into nanotechnology."

The paper this article is based on talks about a ratio between GDP and R&D, the return of overseas scholars and other factors. The paper then details research that looks like what we're trying to do here--except they're doing it a lot more... scientifically. They even call it scientometric analysis! I'm going to steal that...

Where I am using crude statistics available for free and for fun, they use "indicators like total publications, total citation rates, percentage of world share of citations, as well as the top one percent of most highly cited papers in order to measure scientific output."

Something else I'm going to have to look at--"For the input indicators, we used the OECD's Main Science and Technology Studies published online and in print (OECD 2004)." Like me, they experienced difficulty in searching large databases of publications. They found their own solution, but it won't help me going forward.

Some interesting findings:

• Between 1999 and 2004, China moved from 10th to 5th place in terms of overall scientific publications
• The overall US share has dropped from 35% to 30% between 1992 and 2004 (but if the overall number is rising, who cares?)
• Countries where English is commonly spoken (and written, obviously) appear to have an advantage when it comes to having your paper cited by others
• More than 4,400 science and technology journals were published in China in 2001
• Half a million articles are published annually in Chinese journals
• It appears that Chinese scientists read the work of scientists in other countries, but that international scientists don't read (or at least don't cite) work published in Chinese journals (with the exception of some Chinese journals that are published in English)
• The Chinese government declared nanotechnology a critical R&D priority in 2001, and a group of commissions and ministries issued a strategic plan for development of nanotechnology through to 2010
• Definitions are tough. It was hard for the researchers to find all nano-related papers, and only nano-related papers. (I know what that's like--and what it means. You actually have to read, instead of count, and that takes time)
• The paper's authors use really, really cool graphs to show their work. I am extremely jealous

Now I just have to find out where they got those numbers at the end of the news release...  they could have listed those 85 journals...

Human Knowledge and Mars

(This part is repeated from the previous post): NASA and the Smithsonian Institute maintain a database called the Astrophysics Data System. Harvard University is kind enough to host it here. Available to you and I is an abstracting service which will return brief descriptions of articles found by keyword searches. You see where I'm going here?

For a description of coverage and limitations of the database, click here. Then, let's have some fun.

A search for the word Mars in titles and abstracts performed on April 21 2007 returned 20,978 results.

(This is where the new stuff starts): Here we report on 5-year periods going back to 1900. We show annual percentage growth on the previous 5-year total. It isn't pretty--much more like real life than any smooth logarithmic curve. In 22 five-year periods, we show an annual percentage growth above 14% (shown in bold) in only 3 five-year periods(you need a 14.87% annual growth to double every five years). In 7 of the five-year periods, there is a decline in publications (inefficient post, wars, depressions, Mars being further away... lots of logical reasons why this might have occurred).

The serious question is, if our studies of Mars, which have benefited from major government funding from several large governments, and have also benefited from being topical, exciting, and having large sources of data available recently, do not show exponential growth, where would we expect to actually find a doubling of human knowledge every five years? If not here, where?

What I think it shows me is that when there is a lot of funding, real news, a research goal, and the availability of new and important data, we focus on the subject, do the research and improve knowledge. But for extended periods, things just sort of tick along, improving respectably in many years, but showing declines in others. What do you make of the following?

Click if you want to see the numbers.

Human Knowledge and Mars

NASA and the Smithsonian Institute maintain a database called the Astrophysics Data System. Harvard University is kind enough to host it here. Available to you and I is an abstracting service which will return brief descriptions of articles found by keyword searches. You see where I'm going here?

For a description of coverage and limitations of the database, click here. Then, let's have some fun.

A search for the word Mars in titles and abstracts performed on April 21 2007 returned 20,978 results. Let's pretend for the sake of argument that this represents all human knowledge ever gained about Mars. (This may be as close to the truth as we will ever get on this weblog when searching for information about fields of study--but it is obviously not true, so take it with a grain of salt.)

What is the CAGR (Compound Annual Growth Rate) in publications? Well, if we were really strict, we would measure from the date of the first document, which is... 1644. That would give us a CAGR of 2.32% and we would all get very depressed. But a more realistic starting point would be 1830, when regular publications started. This would remove only 10 documents from our total, and would yield a CAGR of 5.78%. That's much better, but far short of doubling every five years. (At that rate, human knowledge about Mars would double every 12.34 years.) Nonetheless, half of the documents in the database have been published since 1993. (Hey--that doubled in 14 years... is the rate of doubling slowing down?)

Just the metadata I got from the site is fascinating, and I'll be referring to it later, looking for short term bursts and lags (and wondering if Mars' distance from the Earth is one of the causes, for example). But here's some interesting information.

The most recent, published in April 2007, is titled "Subsurface Radar Sounding of the South Polar Layered Deposits of Mars and written by Plaut, Jeffrey J.; Picardi, Giovanni; Safaeinili, Ali; Ivanov, Anton B.; Milkovich, Sarah M.; Cicchetti, Andrea; Kofman, Wlodek; Mouginot, Jérémie; Farrell, William M.; Phillips, Roger J.; and 14 coauthors

Document #3001 is titled "Review of the trajectory and atmospheric structure reconstruction for Mars Pathfinder", written by Withers, Paul; Towner, Martin; Hathi, Brijen; Zarnecki, John

Document #6001 is titled "History and perspectives of scientific ballooning," by Frank Lefevre. Hmm. Well, document #6002 is "Stability and evolution of the climate system of Mars," written in 2001 by Stability and evolution of the climate system of Mars

Document #9001 is titled "Mobilization of dust on the Mars surface by the impact of small cosmic bodies," written in 1997 by Rybakov, V. A.; Nemtchinov, I. V.; Shuvalov, V. V.; Artemiev, V. I.; Medveduk, S. A.

Document #12001 is titled "Dry Carbonate Formation on Mars: A Plausible Sink for an Early Dense CO2 Atmosphere?", written in 1990 by Stephens, S. K.; Stevenson, D. J.

Document #15001 is titled "Stratigraphic relationships within Martian polar CAP deposits" written in 1982 by Howard, A. D.; Cutts, J. A.; Blasius, K. R.

Document #18001 is titled "The major Martian dust storms of 1971 and 1973" written in 1974 by Martin, L. J.

The very first document, written in 1644 by Grandami, Jacques, is titled "Nova demonstratio immobilitatis terrae petita EX virtvte magnetica."

If that's a little dense for you, the second, written in 1672 by Flamstead, John, is "An Extract of Mr. Flamstead's Letter Written to the Publisher from Derby Novemb. 16. 1672. Concerning the Appulses, by Him Calculated for A. 1673. of the Moon, and the Other Planets to Fixed Stars; Together with an Observation of the Planet Mars, Made by the Same"

Human Knowledge and Mars

I off-handedly chose Mars as a subject for investigation, basically because there have been several missions to the Red Planet in the past 5 years. Those missions were expensive, they were designed to gather knowledge about Mars... so if human knowledge is doubling every 5 years in any given sector, we might imagine that one of the sectors is Mars. Well, I actually have an ulterior motive, but that's for the end of this post.

The Exploration of Mars is beautifully detailed on Wikipedia. The first probe to actually get close to Mars was in 1962, the Soviet mission Beta Nu 1. It was followed by the US Mariner 4 in 1964. So we have been collecting data from space missions for 45 years. The 1996 Mars Global Surveyor effectively returned more data than all previous missions combined, and the data is available to the public.

The Mars Pathfinder landed on Mars on July 4, 1997, and a remote controlled rover called Sojourner decamped from the lander and went walkabout. Pathfinder returned 16,500 images from the lander and 550 images from the rover, as well as 15 chemical analyses of rock and soil and extensive data on winds and weather. This was followed by a spate of unsuccessful attempts to return to Mars.

In 2001 the run of bad luck ended when NASA's Mars Odyssey orbiter arrived. Its mission is to use spectrometers and imagers to hunt for evidence of past or present water and volcanic activity on Mars. In 2002, it was announced that the probe's gamma ray spectrometer and neutron spectrometer had detected large amounts of hydrogen, indicating that there are vast deposits of water ice in the upper three meters of Mars' soil within 60° latitude of the south pole.

NASA sent a pair of twin rovers toward the planet as part of the Mars Exploration Rover Mission. On 10 June 2003, NASA's MER-A (Spirit) Mars Exploration Rover was launched. It successfully landed in Gusev Crater (believed once to have been a crater lake) on 3 January 2004. It examined rock and soil for evidence of the area's history of water. On July 7, 2003, a second rover, MER-B (Opportunity) was launched. It landed on 24 January 2004 in Meridiani Planum (where there are large deposits of hematite, indicating the presence of past water) to carry out similar geological work.

Mars Reconnaissance Orbiter is a multipurpose spacecraft designed to conduct reconnaissance and exploration of Mars from orbit. The $720 million USD spacecraft was built by Lockheed Martin under the supervision of the Jet Propulsion Laboratory, launched August 12, 2005, and attained Martian orbit on March 10, 2006.

The MRO contains a host of scientific instruments such as the HiRISE camera, CRISM, and SHARAD. The HiRISE camera is used to analyze Martian landforms, whereas CRISM and SHARAD can detect water, ice, and minerals on and below the surface. (In case you haven't guessed, I'm just copying from Wikipedia here.)

Now, the Wikipedia article on Mars itself is quite long. It contains very little material that could have been written before the past 45 years--the planet was mapped in 1840, some basic astronomical information has been long known, but almost all of what Wikipedia thinks is important is recent, and most of it really, really recent. Of the 79 citations at the bottom of the article, 30 were written after 2004.

So I'm ready to admit that human knowledge about Mars has doubled every 5 years since 1962. I'll be searching for more evidence to back this up. But here's my ulterior motive: I think that new sectors, or sectors that have big money thrown at it, or sectors where there is a clear public good to be obtained from applied research, might well indeed experience exponential increases in knowledge. But are they sustained? Does knowledge gained transfer readily to similar sectors? (Do we, for example, know as much about Venus or Mercury?) If new knowledge gained in the past 5 years about Mars (and I include getting to Mars, data collection design, etc.) leads to similar increases in knowledge about other planets in our solar system, then I think we're on to something here.

Wow!

I was about to click out of the Lawrence Livermore Laboratory's website when I stumbled across this news release: "Lawrence Livermore National Laboratory’s institutional “Grand Challenge” scientific computing program has allocated 83.7 million CPU hours to 17 research projects ranging from astrophysics, chemistry, materials and biosciences to earthquake and climate simulation on Laboratory supercomputers."

... "Research projects were selected by the Laboratory’s Deputy Director for Science and Technology, the Laboratory Strategic Program Board and the Laboratory Science and Technology Office. To be considered, proposals had to “address a grand-challenge-scale, mission-related problem that promises unprecedented discoveries in a particular scientific and/or engineering field of research, and if successful, will result in high-level recognition by the scientific community at large.”

... "For example, a multi-institutional group of climate scientists will conduct the most detailed multi-decade global climate simulations ever performed as part of the larger effort to understand how climate changes over time. “By performing these climate simulations of unprecedented detail and fidelity and making them available to the climate research community, we have the potential to dramatically advance our understanding of climate and how it may be affected by anthropogenic factors,”

Grand Challenge recipients have been allocated time on Atlas, the new 44 teraFLOP (trillion floating operations per second) machine and Thunder, a 22 teraFLOP machine -- systems dedicated to unclassified research through the Laboratory’s Multi-programmatic and Institutional Computing program. Central processing unit or CPU time is measured across the multiple CPUs in a computer. For example, two CPU hours can be one CPU used for two hours or two CPUs used for one hour. High performance computers generally consist of thousands of CPUs; the Atlas system contains 9, 216 CPUs."

You know, it's stories like this that make me optimistic about the future.  "Over the last 10 years, high performance computing resources dedicated to unclassified institutional research have increased more than a thousand fold from 72 giga (72,000 million) FLOPS in 1997 to 81 teraFLOPS today."

This is what the Singularity protagonists are pointing to. Prosaic sad sacks like me are stubbornly waiting to see the results of the research performed. Something inside me grumbles that a well-designed program wouldn't need 82 million hours to execute... (this from someone who's never written a line of code in his life).

On the other hand, there are specific sectors where this exercise in and of itself might result in the doubling of human knowledge.