Does Human Knowledge Double Every 5 Years?

A Smart Science Fiction Writer Looks At The Future--And The Singularity

Charlie Stross looks at improvements in the speed of travel, information technology and bandwidth, says some very interesting things about convergence and the near future.

One idea just charms me--that our children will never have to worry about being lost, any more than we had to worry about being eaten by wolves. I suppose in about ten minutes I'll start thinking about the Big Brother implications, but these ten minutes are nice.

Hat tip Brad DeLong.

Programming, Software and Computer Science

Computer programming has a lot of work and a lot of people centered around it. If the Singularitarians are correct, we're going to need them all.

After all, what good is it if computing speeds and memory keep doubling if we cannot program computers to do what we want them to do? And the Singularitarians want computers to do some pretty ambitious things. In fact, given their wish list, it seems clear that human knowledge of computer programming has to double every five years if we are going to get the magic box to do what we want it to do. We will need new programming languages, tighter coding, better algorithms, and quite possibly a new way of looking at programming. Is this happening?

We've seen elsewhere that the number of people engaged in computer sciences has grown to 180,000 new entrants per year in the U.S. alone--and considering the base date for this field, that is evidence of healthy growth. Let's use Scirus to see what this growth in professionals has produced in the way of patents and journal articles.

Preliminary results: between 2002 and 2006, 38,685 journal articles were published with the word 'programming' in the title or abstract. As the total is 145,404, only 26.6% of all journal articles were published in the last five years. Of the 367,971 patents noted on Scirus with 'programming' in the title or description, 201,548, or 54.7% were published between 2002 and 2006--so that has doubled. Not the first time we have seen differing growth rates for journals and patents.

For 'software,' the period 2002-2006 saw 224,696 journal articles published, 46% of the total of 486,262. For patents, as with 'programming,' the period was more robust--589,919 out of a total of 900,846, or 65% of the total.

The base year for 'programming' is 1940, when a UK patent was filed for improvements on a statistical calculating device. The compound annual growth rate for journal publications is 12.55%--good, but not good enough to double every five years (it is less than six years, to be fair). The compound annual growth rate for patents is 12.83%.

Remember that we need a CAGR (Compound Annual Growth Rate) of 14.87% to double every five years.

For 'software, the base year is 1965, when 5 journal articles were published and 1 patent was filed. And software is a winner, with CAGR of 22.83% for journal publications!  Hooray for coders everywhere! And a double huzzah to IBM, Microsoft, Sun and countless others, as patent growth since 1965 has a CAGR of 33.53%. Another winner!

As there are so many more patents than journal publications for both search terms, we should probably give more weight to patents, and be extremely pleased that human knowledge appears to have doubled in the past five years in this area, especially given the gloomy results reported in my previous post last week.

Click below to see results and analysis.

Outcomes

Advances in human knowledge should make life better. That's hard to measure. However, we can measure if human life gets longer (it does), healthier, richer (if not more rewarding), etc.

The rest of this post is philosophizing--feel free to skip. And in a previous post, I talk about the Outcomes Fallacy. But if you want thinking on the issue of human knowledge and outcomes, click to read more.

Sobering News

The post below from yesterday, which details the stately, rather than exponential, increase in granted patents in the U.S., has really got me thinking, and I think we'd better report on the state of play here.

Let's detail those areas where we have seen adequate growth to double human knowledge every 5 years:

Update: We have since found other areas where the CAGR is sufficient to double human knowledge every five years:

Update 2: A modest review revealed errors in my calculations regarding oncology. Its position has changed (and not for the better).

Prions, where academic journal publications have increased at a CAGR of 25.57%, and patents have increased by 33.76%

Software, where academic journal publications have increased at a CAGR of 22.83% and patents have increased by 33.53%

Post-graduate enrollment in China, which has increased at a CAGR of 22.38% (although see below for more discouraging information about the growth of tertiary education worldwide)

(All percentages are CAGR (compound annual growth rate.)

1. Nanotechnology, with a blistering growth in publications of 97.16% per year. Patents, 76.15%.

2. Global Warming publications, with a of 16.85%

3. Total Chinese patent applications, at 25% per annum

4. Chinese computing patents, 58.29%

5. Chinese telecommunications patents, 30.26%

(was oncology)

7. Epidemiology patents, 22.76%

So far, that's it.

Now let's look at where we did not find anything like doubling every five years (remembering that we need a CAGR of 14.87% to double in five years time):

Update: We have now found other areas where human knowledge does not appear to be growing at a rate that will double in five years time:

Worldwide university enrollment, which has a CAGR of 4.85%

Alzheimer's disease, where academic publications have a CAGR of 9.65% and patents a CAGR of 11.26%

Programming, where academic publications have a CAGR of 12.55% and patents a CAGR of 12.83%

Chinese university enrollment, with a CAGR of 9.76%

1. U.S. patents granted, 5.21%

2. Epidemiology publications, 8.85%

3. Shale oil extraction, publications 5.59%, patents, 2.61%

4. Publications on Mars, 5.78% (see also here and here)

5. U.S. Book publishing, 5.14%

6. Indian university enrolment since 1947, 5.51%

7. Climate Models, 12.66% (Very close to 5-year doubling rate of 14.87%)

8. Improvements in Climate Models, 13.76% (Also very close)

9. Growth in publication of scientific and technical papers, 1995-1999

             OECD 0.1%

             U.S., -1.8%

             Canada, -2.2%

             UK, -0.1%

             Netherlands, -0.5%

10. Publications in astrophysics, 4% (since 1970)

11. Growth in percentage of cancer patients surviving 5 years after diagnosis, 0.7% (1960-2005)

12. Drop in percentage of soldiers dying from wounds received in battle, 1.54% (1941-2006)

13. Projected growth in number of scientists (U.S.), next ten years, 7.18%

14. Patent filings in India, last five years, 8.45%

15. Increase in patent filings, World International Patent Office, 1 year, 6.4%

6. Oncology pubications, 9.23% (revised)

Considering I cherry picked areas that I thought would be most likely to show robust growth, this is scary.

          

But This Isn't Just About The Singularity, Dammit!

The Singularitarians need an exponential growth of human knowledge to enable their theory of short term artificial intelligence.

But I submit they are not the only ones who are changing their behaviour and expectations because of this general idea floating around. The idea that human knowledge doubles every five years can become an excuse for inaction.

If people think that human knowledge doubles every five years, they may think that improved science and technology will provide a 'deus ex machina' last minute cavalry charge over the hill solution to something, meaning they don't have to change their current behaviour.

They might continue smoking, thinking that surely, by the time they're 69 (the average age of diagnosis for cancer), we will have found a silver-bullet solution for all cancers.

Their organisations don't modify damaging behaviour, thinking that we will find a way to turn pollution into ice cream and champagne, or replace depleted fisheries with genetically modified de-boned cod that jump into our nets with a smile.

I think this attitude, to the extent that it exists, is quite dangerous. If my first pass through publicly available data suggests anything (and even when I finish it will need to be approached academically), it suggests that human knowledge does not double every five years.

What we do see is that specific sectors, such as nanotechnology or proteomics, grow even faster than that for short periods of time:

• When the sector is new
• When the sector is sexy
• When government or private industry sees a short term gain resulting from it
• When intermediate goals are visible and attainable

But exponential growth then reverts to a mean, or returns to an average growth that is far slower. That growth rate varies according to the availability of inputs, such as researchers, funding and above all the availability of findings from outside the given sector--the multi-disciplinary rescue. This has rescued Moore's Law more than once, and it is applicable across fields.

Not every problem deserves the resources of a Manhattan Project, which is the prime example of how throwing money and scientists at an issue can produce results. Some problems can be solved without a Manhattan Project. But I fear that problems like lung cancer and artificial intelligence do need resources at that scale...

...and I fear even more that a general, sloppy impression that human knowledge doubles every five years without any real effort or a need to get behind the knowledge machine and push is actually harming the cause of scientific and technological progress. It relieves us of personal responsibility. Oh, the new horde of Chinese researchers will solve it, so we don't have to.

That is pernicious.

What Is The Singularity?

Okay. Here is the definition provided by the Singularity Institute for Artificial Intelligence (SIAI):

"The Singularity is the technological creation of smarter-than-human intelligence. There are several technologies that are often mentioned as heading in this direction. The most commonly mentioned is probably Artificial Intelligence, but there are others: direct brain-computer interfaces, biological augmentation of the brain, genetic engineering, ultra-high-resolution scans of the brain followed by computer emulation. Some of these technologies seem likely to arrive much earlier than the others, but there are nonetheless several independent technologies all heading in the direction of the Singularity – several different technologies which, if they reached a threshold level of sophistication, would enable the creation of smarter-than-human intelligence."

The Institute is described on Wikipedia here. I emailed the founder, Eliezer Yudkowsky, to get his thoughts on what I am doing here. He hasn't responded as yet.

For 'Singularitarians', the Singularity is a big transformative concept, one of those spectacular theories that will change human life as we know it. When we create intelligence greater than our own, we will solve problems that we cannot solve today. Furthermore, many of the technologies (and especially nanotechnology) that will be used to create artificial intelligence will be used by the artificial intelligence to help us with healthcare, energy (PDF), space travel and more. God help us if we create an artificial intelligence that wants to sit on the couch and watch the soaps.

Those who fear artificial intelligence, nanotechnology (and perhaps big transformativc concepts), manage to create a vision of some rough beast slouching towards Bethlehem waiting to be born. From fears of grey goo (nanotechnology run wild) to the destruction of human ambition, AI is a threat--perhaps the biggest outsourcing threat of all, especially if it is embodied in robots.

Nanotechnology and Human Knowledge

There's a search engine called Scirus for scientific information. A search performed on Sunday, 22 April 2007 with the search term 'nanotechnology' returned 637,500 results. Fortunately, we can winnow that down to 28,846 journal results and sort them by date. (We also found reference to 5,148 patents) So, here we go again... 23,489 of those journal articles were published after 2002 ended. That's 81% of the total. Similarly, 4,494 patents were filed since the end of 2002, 87% of the total.

Using 1987 as a base year, we find CAGR in journal publications of 42.03%, which would double in 1.98 years. For patents, we see a CAGR of 44.91%, which would double every 1.87 years.

And this is why Ray Kurzweil and the rest of the Singularity gang get excited. Remember we need annual growth of 14.87% to double in five years. Nanotechnology has got it in spades.

Last year's growth was 37.87%. Maybe they will take over the world... but I will explain later why I think growth in nanotechnology is a necessary, but not sufficent pre-requisite for the Singularity.

Note that the advanced search by year  yielded 28,863 journal results (but it found all 5,148 patents )(one reason these types of databases drive academic researchers crazy--I hear Google Scholar is worse, and I'm afraid I'm going to find out soon...)

Click if you want to see the numbers

Chinese Papers in Nanotechnology

Well, I was going to wait before starting on nanotechnology, but I don't have a convenient place to park this information, so you get it in toto:

From PhyiscsToday.org: "A paper by Ping Zhou and Loet Leydesdorff (PDF) suggests that China is rapidly developing research expertise in nanotechnology. Six years ago, US-based scientists published 50% of the papers in the journal Nanotechnology and China accounted for zero. Today 25% of papers in the journal are from US-based scientists and 15% from China. Zhou and Leydesdorff report that between 1999 and 2005, for 85 journals in the fields of chemistry, physics and material sciences that are relevant to nanotechnology, the number of papers published by China rose exponentially to 8.34%, while papers published by other countries either remained steady or declined. China's overall world share in science journals is 6.52%, suggesting that China is aiming to be at the forefront of research into nanotechnology."

The paper this article is based on talks about a ratio between GDP and R&D, the return of overseas scholars and other factors. The paper then details research that looks like what we're trying to do here--except they're doing it a lot more... scientifically. They even call it scientometric analysis! I'm going to steal that...

Where I am using crude statistics available for free and for fun, they use "indicators like total publications, total citation rates, percentage of world share of citations, as well as the top one percent of most highly cited papers in order to measure scientific output."

Something else I'm going to have to look at--"For the input indicators, we used the OECD's Main Science and Technology Studies published online and in print (OECD 2004)." Like me, they experienced difficulty in searching large databases of publications. They found their own solution, but it won't help me going forward.

Some interesting findings:

• Between 1999 and 2004, China moved from 10th to 5th place in terms of overall scientific publications
• The overall US share has dropped from 35% to 30% between 1992 and 2004 (but if the overall number is rising, who cares?)
• Countries where English is commonly spoken (and written, obviously) appear to have an advantage when it comes to having your paper cited by others
• More than 4,400 science and technology journals were published in China in 2001
• Half a million articles are published annually in Chinese journals
• It appears that Chinese scientists read the work of scientists in other countries, but that international scientists don't read (or at least don't cite) work published in Chinese journals (with the exception of some Chinese journals that are published in English)
• The Chinese government declared nanotechnology a critical R&D priority in 2001, and a group of commissions and ministries issued a strategic plan for development of nanotechnology through to 2010
• Definitions are tough. It was hard for the researchers to find all nano-related papers, and only nano-related papers. (I know what that's like--and what it means. You actually have to read, instead of count, and that takes time)
• The paper's authors use really, really cool graphs to show their work. I am extremely jealous

Now I just have to find out where they got those numbers at the end of the news release...  they could have listed those 85 journals...

Wow!

I was about to click out of the Lawrence Livermore Laboratory's website when I stumbled across this news release: "Lawrence Livermore National Laboratory’s institutional “Grand Challenge” scientific computing program has allocated 83.7 million CPU hours to 17 research projects ranging from astrophysics, chemistry, materials and biosciences to earthquake and climate simulation on Laboratory supercomputers."

... "Research projects were selected by the Laboratory’s Deputy Director for Science and Technology, the Laboratory Strategic Program Board and the Laboratory Science and Technology Office. To be considered, proposals had to “address a grand-challenge-scale, mission-related problem that promises unprecedented discoveries in a particular scientific and/or engineering field of research, and if successful, will result in high-level recognition by the scientific community at large.”

... "For example, a multi-institutional group of climate scientists will conduct the most detailed multi-decade global climate simulations ever performed as part of the larger effort to understand how climate changes over time. “By performing these climate simulations of unprecedented detail and fidelity and making them available to the climate research community, we have the potential to dramatically advance our understanding of climate and how it may be affected by anthropogenic factors,”

Grand Challenge recipients have been allocated time on Atlas, the new 44 teraFLOP (trillion floating operations per second) machine and Thunder, a 22 teraFLOP machine -- systems dedicated to unclassified research through the Laboratory’s Multi-programmatic and Institutional Computing program. Central processing unit or CPU time is measured across the multiple CPUs in a computer. For example, two CPU hours can be one CPU used for two hours or two CPUs used for one hour. High performance computers generally consist of thousands of CPUs; the Atlas system contains 9, 216 CPUs."

You know, it's stories like this that make me optimistic about the future.  "Over the last 10 years, high performance computing resources dedicated to unclassified institutional research have increased more than a thousand fold from 72 giga (72,000 million) FLOPS in 1997 to 81 teraFLOPS today."

This is what the Singularity protagonists are pointing to. Prosaic sad sacks like me are stubbornly waiting to see the results of the research performed. Something inside me grumbles that a well-designed program wouldn't need 82 million hours to execute... (this from someone who's never written a line of code in his life).

On the other hand, there are specific sectors where this exercise in and of itself might result in the doubling of human knowledge.

One... Singular Dilemma

Singularitarians say that computing will advance to the stage where artificial intelligence outstrips human capabilities and then will happily begin designing even better artificial intelligence. I hope they're right.

But hand on heart, I honestly believe that if we had the magic box (or network of boxes) that could compute as quickly and efficiently as it will in 2035 (one of the dates proposed for arriving at the Singularity), we couldn't make it work. We couldn't program it to do what we expect AI to do at the time of the Singularity.

Human knowledge will have to increase in order to make the box work. I suspect (and am searching for evidence) that it will have to increase exponentially. (The alternative is finding a genius, which could happen at any time, but we can't count on it. That genius may exist, but have died yesterday in Darfur or Baghdad.)

If human knowledge is not increasing exponentially (or if we don't find and train the right genius), I don't think the Singularity can happen. Ever.