Does Human Knowledge Double Every 5 Years?

How Would We Define Victory?

Well, time to define a few things here. We are trying to discover if human knowledge doubles every 5 years. To do this, we are testing three hypotheses:

1. We learn so much new information that the actual stock of knowledge is double what it was 5 years ago. We are testing this by seeing if scientific publications and patents are doubling every 5 years. We also look and see if there are twice as many scientists and researchers as 5 years ago.
2. That new and old human knowledge is accessible and usable to twice as many people as it was 5 years ago. We're looking at things like the size and reach of websites like Wikipedia, and also referring to our search for more creators and users of information.
3. We are disproving incorrect hypotheses at such a rate that we are doubling human knowledge by unlearning untruths, which corrects mistakes and frees up time for productive learning.

But this is a bit simplistic.  If I know one thing and learn another, have I doubled human knowledge or just what I have learned? Is a learned fact human knowledge before it is published?

If I know one thing and, without learning another, communicate that one thing to another human being, does that double human knowledge because two of us now know it? Or is that just repetition?

If I discover an hypothesis is not correct, does it make any difference if others continue to hold the old and incorrect belief?

And then we must make things difficult by defining knowledge. Many of you will have seen the pyramid that has data at the bottom, information on top of it, then knowledge and then wisdom. The implication is that data is a fact revealed, information is the fact in context, knowledge is is the fact absorbed and assimilated, and wisdom is right action based on knowledge as experienced. But is that an accurate or adequate definition of knowledge? Is it sufficient?

Wikipedia has a definition--it's a bit long, but the start is promising. Of course the start is a quote from Oxford's English Dictionary. "Knowledge is defined (Oxford English Dictionary) variously as (i) facts, information, and skills acquired by a person through experience or education; the theoretical or practical understanding of a subject, (ii) what is known in a particular field or in total; facts and information or (iii) awareness or familiarity gained by experience of a fact or situation." Yeah, that's it.

So does learning that the Earth is more than 6,000 years old mean anything if millions don't believe it? If creationists make learned scientists spend time in court patiently explaining evolution instead of working in the laboratory--their true 'field of the Lord,' have we learned anything? Does knowledge have to be unanimously held, or unanimously available?

Boy, where's a philosopher when you need one?

Global Warming and Hypothesis 3

If you've read the beginning days of this weblog (which today celebrates two weeks of actual existence), you might recall that I am trying to test three hypotheses. The third is that we are disproving (or falsifying) incorrect assumptions, hypotheses and opinions at a rate that effectively doubles human knowledge (knowing something is not true is really, really important). This is just as important in global warming as with anything else.

Critics of global warming have for years pointed to the fact that satellite observations, taken from a single dataset, showed slight cooling in the tropical troposphere since 1979. Since this conflicted with what climate models predict, this was a serious problem for those attempting to warn the public about global warming.

So they went and got more data, which is what you're supposed to do. This 2005 news release from the Lawrence Livermore Lab (close to where I grew up), shows that further research found better data that agrees with modern climate models.

As there is still considerable controversy about global warming, there's a whole lot more falsifying and disproving that needs to be done, probably on both sides of the fence. If you think about it, the  beneficiaries of this disproof of the previous data are the critics of global warming as well as those who strongly believe, as critics now actually have to examine their assumptions, look for more or better data, or find explanations for what this research showed.

Happy Coincidence

A chart I put up on the weblog cited Derwent's World Patents Index as a source. It is found at Thomson.com, and it has information regarding 31.66 million patent documents. (Okay, let's future proof this--in 2012, will it have information on 63 million patent documents?)

They charge to get in and look around, so I'm hovering on the periphery, seeing if I can find more meta-data about the database. For example, an article titled 'Open access to the medical literature: How much content is available in published journals?' by Marie McVeigh and James Pringle, published in 2005, indicates that about 20% of medical literature published in 174 journals since 1992 is 'open access', that is, freely available online. They estimate that a bit less than 60% of medical literature in that time period can be accessed electronically. We also find that Thomson's ISI Web of Knowledge has 20 million users from 81 countries accessing a database of 22,000 journals, 23 million patents and much more. All good supporting data for my second hypotheses, about the increasing availability of human knowledge. But I would love to see the growth rate for new data coming into that database, and I can't find it.

However, I did find this--an article called 'Eastward ho! --The geographic drift of global R&D' written in March 2007 by Bob Sternbridge. It touches on a number of topics I've written about below, and has some interesting numbers:

• The number of computing patents filed in China has risen from 2,000 to 20,000 in five years! Clear doubling in the right time frame. (Pity the increase was not replicated in Japan or Korea, which were also included in the study)
• The number of communications patents filed in China has also risen in the same time frame, from 8,000 to 30,000--again a doubling! Sadly, again not replicated in Japan or Korea.
• More importantly, the total number of Chinese patent applications is growing by 25% per annum. But the article then says that Japan and US patent filings have been flat or declining in recent years. And a lot of the patent filings are by multinational corporations
• R&D spending in China has risen from 89.6 billion yuan in 2000 to 245 billion yuan between 2000 and 2005
• The number of researchers in China grew 77% over the ten years from 1995 to 2004

I think it will take me a couple of hours to digest this and incorporate it into previous posts. More later.

There Were Hints And Allegations

Remember we are testing three hypotheses here:

1. The actual store of human knowledge (not, repeat not, data) has doubled in the past 5 years

2. Access to knowledge has grown so quickly that it fulfills the same function as a doubling of human knowledge

3. We are disproving previously held theories at such a rate that it frees up resources to concentrate on useful lines of enquiry resulting in an effective doubling of knowledge.

The easiest to find positive indications for is number 2. As an example, 5 years ago:

• Wikipedia had 80,000 articles and 125,000 hits per day in August 2002. On January 1, 2007, it had over 6 million articles (in 251 languages!) and in Dec. 06 it averaged more than 5 million daily unique visits. Okay, that's more than double... but this only works for hypothesis number 2, as we have no indication on how much of the knowledge placed on Wikipedia during that time period existed before 2002.

It Should Be Lumpy, Right?

We wouldn't expect a uniform doubling of human knowledge within a 5-year timeframe, right? Even if this is a law as inexorable as Moore's (and that doesn't seem as inexorable as it once was, or am I reading the Dell ads wrong?), one would reasonably expect that human knowledge would advance a little in some areas and a lot in others. You with me?

Given what the world's attention has been focused on since 2002, here are some of the areas where I think human knowledge should have increased fastest:

• Epidemiology, thanks to SARS and Avian flu
• Biofuels
• Vaccines
• Proteome
• Oil extraction from rock and shale
• Mars
• Genetically Modified Organisms
• Fishery management
• Global warming
• Solar activity
• The Long Tail
• Stem cell cultivation
• Infectious diseases

I will treat this as a proxy list of subject areas to explore. I'm going to be digging around to see if we've seen dramatic advances in any or all of these. I'll probably dedicate a few posts to each of them. Got any suggestions?