Natural language processing was ( Re: How best to move system to SSD)
Terry Duell wrote:
Hope that helps somewhat at least ;-) Don't think so :-( I'll keep searching for clues. The way Google seems to work these days one has to be quite enlightened to figure out the right search string to get an answer that helps.
Obviously natural language processing must be an extremely difficult problem, if Google with all their expertise and processing power can't put a natural language, front end on their search engine. I wonder whether some entrepreneurial start-up company with a new idea; might put together software which disambiguates say short English language search questions via a dialogue; then compiles a complex Google search string, to achieve that result ? regards Rohan McLeod
Rohan McLeod <rhn@jeack.com.au> wrote:
Obviously natural language processing must be an extremely difficult problem,
That depends on what kind of processing you want to do and what results you hope to achieve from it. At a deeper level, there are still foundational disagreements about how best to construct a systematic theory of the semantics of natural languages. Important progress has been made in this area over the last century, however, which is very encouraging. (I wrote my PhD in this area).
if Google with all their expertise and processing power can't put a natural language, front end on their search engine.
I suspect they've concluded that their ranking and query analysis algorithms already employ the most effective techniques available. I have no doubt that the designers of Google Search are well versed in the literature of natural languge processing - Google employ PhD graduates in large numbers.
I wonder whether some entrepreneurial start-up company with a new idea; might put together software which disambiguates say short English language search questions via a dialogue; then compiles a complex Google search string, to achieve that result ?
They would have to perform consistently better than Google's own query processing algorithms to be worthwhile though.
Jason White wrote:
Rohan McLeod <rhn@jeack.com.au> wrote:
.....................snip They would have to perform consistently better than Google's own query processing algorithms to be worthwhile though.
Jason; just so I'm sure we are talking about the same thing; the issue I am concerned with is translating a natural language search query, into the Google search string, which most economically returns the required result. Not setting up some alternative to Google's search algorithm ! Yes ? regards Rohan McLeod
Rohan McLeod <rhn@jeack.com.au> wrote:
Jason; just so I'm sure we are talking about the same thing; the issue I am concerned with is translating a natural language search query, into the Google search string, which most economically returns the required result. Not setting up some alternative to Google's search algorithm ! Yes ?
I think we're talking about the same thing. It would need to be shown that some other form of language processing can consistently perform better than just supplying your "natural language" query as input to the existing search engine.
Jason White wrote:
Rohan McLeod <rhn@jeack.com.au> wrote:
Jason; just so I'm sure we are talking about the same thing; the issue I am concerned with is translating a natural language search query, into the Google search string, which most economically returns the required result. Not setting up some alternative to Google's search algorithm ! Yes ? I think we're talking about the same thing. It would need to be shown that some other form of language processing can consistently perform better than just supplying your "natural language" query as input to the existing search engine. Yes certainly; but surely the irrelevance of many search results indicates improvement is possible ?
regards Rohan McLeod
Rohan McLeod wrote:
Jason White wrote:
I think we're talking about the same thing. It would need to be shown that some other form of language processing can consistently perform better than just supplying your "natural language" query as input to the existing search engine.
Yes certainly; but surely the irrelevance of many search results indicates improvement is possible ?
Seems to me that simply indicates that the incentive of most principals is to return "irrelevant" results. Even excluding SEOs, Google itself would return nothing but ads if that wouldn't drive away their product.
Rohan McLeod <rhn@jeack.com.au> wrote:
Yes certainly; but surely the irrelevance of many search results indicates improvement is possible ?
There's an entire field of research devoted to "information retrieval" systems, the aim of which is to return the most relevant search results in response to a query. Google has considerable expertise in this area - in fact, as I recall, Google's founders were conducting information retrieval research at Stanford, and the company grew out of their research project. What I'm suggesting, then, is that further improvement may require the creation of algorithms that haven't yet been created (advances in computational linguistics, database technology, information retrieval etc.).
Jason White wrote:
Rohan McLeod <rhn@jeack.com.au> wrote:
Yes certainly; but surely the irrelevance of many search results indicates improvement is possible ? There's an entire field of research devoted to "information retrieval" systems, the aim of which is to return the most relevant search results in response to a query. Google has considerable expertise in this area - in fact, as I recall, Google's founders were conducting information retrieval research at Stanford, and the company grew out of their research project.
What I'm suggesting, then, is that further improvement may require the creation of algorithms that haven't yet been created (advances in computational linguistics, database technology, information retrieval etc.).
I would be interested to hear about the thesis of your Ph D; and your ideas concerning the above problems regards Rohan McLeod regards Rohan McLeod
Rohan McLeod <rhn@jeack.com.au> wrote:
I would be interested to hear about the thesis of your Ph D; and your ideas concerning the above problems
My research is concerned with basic questions about how to construct a semantic theory of any given language. There are foundational disagreements in this area concerning the limits of semantic analysis, which I seek to address by investigating Robert Brandom's "inferentialist" theory of meaning. I think modern formal logic and philosophy have made real progress in analyzing and specifying the semantics of natural language. Some of this insight has been applied to computational problems, but there is much progress remaining to be made on both the theoretical side and in computational applications. Search engines use statistical techniques rather than logic-based semantics to process queries.
Jason White wrote:
Rohan McLeod <rhn@jeack.com.au> wrote:
I would be interested to hear about the thesis of your Ph D; and your ideas concerning the above problems My research is concerned with basic questions about how to construct a semantic theory of any given language. There are foundational disagreements in this area concerning the limits of semantic analysis, which I seek to address by investigating Robert Brandom's "inferentialist" theory of meaning.
So would this have implications for the problem of constructing 'the meaning of a sentence'; given the meaning of it's constituent words ?
I think modern formal logic and philosophy have made real progress in analyzing and specifying the semantics of natural language. Some of this insight has been applied to computational problems, but there is much progress remaining to be made on both the theoretical side and in computational applications.
Do you think that the search engine problem might be broken into two parts: 1/ A precise symbolic way of expressing a complex search pattern; say analogous to a 'regular expression' formula 2/ A 'short, simple sentence; natural language processing' algorithm; which then constructs sequences of such symbolic search formulas; equivalent to the disambiguated natural language sentence ?
Search engines use statistical techniques rather than logic-based semantics to process queries.
Do you see a place for neural-nets in this area ? regards Rohan McLeod
Rohan McLeod <rhn@jeack.com.au> wrote:
So would this have implications for the problem of constructing 'the meaning of a sentence'; given the meaning of it's constituent words ?
Yes, exactly.
Do you think that the search engine problem might be broken into two parts:
1/ A precise symbolic way of expressing a complex search pattern;
say analogous to a 'regular expression' formula
2/ A 'short, simple sentence; natural language processing' algorithm;
which then constructs sequences of such symbolic search formulas; equivalent to the disambiguated natural language sentence ?
How would you then associate this with search results? It wouldn't be feasible to parse and analyze all of the text of a large collection of Web pages, so far as I am aware.
Search engines use statistical techniques rather than logic-based semantics to process queries.
Do you see a place for neural-nets in this area ?
They're among the probabilistic techniques available for application, yes. However, neural networks are well outside my field.
Jason White wrote: ...........snip
Do you think that the search engine problem might be broken into two parts: 1/ A precise symbolic way of expressing a complex search pattern; say analogous to a 'regular expression' formula 2/ A 'short, simple sentence; natural language processing' algorithm; which then constructs sequences of such symbolic search formulas; equivalent to the disambiguated natural language sentence ?
How would you then associate this with search results? It wouldn't be feasible to parse and analyze all of the text of a large collection of Web pages, so far as I am aware.
I quite agree it is certainly not currently feasible to "parse and analyze all of the text of a large collection of Web pages" The aim would simply be to decrease the number and increase the relevance of search results; whilst ensuring the search was as exhaustive as possible. Jason many thanks for sharing your interests in language; regards Rohan McLeod
participants (3)
-
Jason White -
Rohan McLeod -
Trent W. Buck