The Turing test has been used as a standard for measuring artificial intelligence by comparing it to human conversational characteristics. Soon, we will need to move beyond that standard and measure intelligence by itself. Two researchers are working on a mathematical model to accomplish that.
On the hunt for universal intelligence – [physorg.com]
How do you use a scientific method to measure the intelligence of a human being, an animal, a machine or an extra-terrestrial? So far this has not been possible, but a team of Spanish and Australian researchers have taken a first step towards this by presenting the foundations to be used as a basis for this method in the journal Artificial Intelligence, and have also put forward a new intelligence test.
Anytime Universal Intelligence – [dsic.upv.es]
SUMMARY OF THE PROJECT:
Following ideas from the first intelligence definitions and tests based on Algorithmic Information Theory [Dowe and Hajek 1997] [Hernandez-Orallo 2000a] [Legg and Hutter 2007], we face the challenge of constructing the first universal, formal, but at the same time practical, intelligence test. The key issue is the notion of “anytime” test, which will allow a quick convergence of the test to the subject’s level of intelligence and a progressively better assessment the more time we provide. If we succeed, science will be able to measure intelligence of higher animals (e.g. apes), humans and machines in a universal and practical way.
Measuring universal intelligence: Towards an anytime intelligence test – [monash.edu.au]
In this paper, we develop the idea of a universal anytime intelligence test. The meaning of the terms “universal” and “anytime” is manifold here: the test should be able to measure the intelligence of any biological or artificial system that exists at this time or in the future. It should also be able to evaluate both inept and brilliant systems (any intelligence level) as well as very slow to very fast systems (any time scale). Also, the test may be interrupted at any time, producing an approximation to the intelligence score, in such a way that the more time is left for the test, the better the assessment will be. In order to do this, our test proposal is based on previous works on the measurement of machine intelligence based on Kolmogorov complexity and universal distributions, which were developed in the late 1990s (C-tests and compression-enhanced Turing tests). It is also based on the more recent idea of measuring intelligence through dynamic/interactive tests held against a universal distribution of environments. We discuss some of these tests and highlight their limitations since we want to construct a test that is both general and practical. Consequently, we introduce many new ideas that develop early “compression tests” and the more recent definition of “universal intelligence” in order to design new “universal intelligence tests”, where a feasible implementation has been a design requirement. One of these tests is the “anytime intelligence test”, which adapts to the examinee’s level of intelligence in order to obtain an intelligence score within a limited time.