A few days back I was reading an interesting paper by Pat Langley, “The changing science of Machine Learning”. It was a very interesting look at the humble beginnings of machine learning and where it has reached now and what has been missed. Some of the cobwebs in my mind have been cleared, thanks to him. I really agree with him when he says that machine learning has got more into the data analysis mumbo-jumbo ignoring the challenges of reasoning and problem-solving. Another point he mentions is of the fact that only papers with mathematical formalization are thought to carry weight in a premier machine learning conference. I always thought that I might have to change lanes at some point in time w.r.t my research which I shall explain shortly.
My work is related to integrating knowledge sources to perform different tasks, and devise an intelligent mechanism for doing the integration instead of an ad-hoc approach. The problem may be unclear but that is not the point I am trying to make. I was always under an impression that this problem does not have the mathematical rigor, and I may have to switch into something more mathematically challenging. My pitfall was that I mixed up ‘mathematically challenging’ and ‘challenging problems’ and almost forgot that challenging problems are a superset of mathematically challenging problems.
The other point I wish to make is the interesting cycle of ‘history repeats itself’ albeit in a different way. In the 1980′s the emphasis is on knowledge-based approaches which paved the way for statistics based approaches in the 90′s in the case of machine learning. The main reason for this is the bottleneck in acquiring knowledge sources. Now, especially in the case of text-related machine learning tasks, knowledge-based approaches have again come to the fore in the late 2000′s thanks to the seminal work done by Evgeniy Gabrilovich and others in using the wikipedia as a background knowledge resource.
So my observation is that irrespective of the path we take to solve a problem, we might think that we change lanes in the short term, but actually we may be converging towards something more general or unified. I talk in terms of text processing but this may be applicable to many other areas. Solving a text processing problem would make us switch lanes by trying out different methods in machine learning, natural language processing or some further specialized methods like case based reasoning, the final goal is to build an intelligent system chasing the elusive goal of ‘artificial intelligence’. We may change the lanes but the paths eventually converge.