Question Generation Technology (QGT) in Deep Learning - Why Defining New Questions Is More Important Than Finding Answers in Data Analytics.
For anyone heavily involved in the development of Deep Learning technology, by far one of the most difficult things we face is actually putting what is in our heads into meaningful words. By this I mean answering the many questions that the inquisitive audience ask of us. For example, I may be asked “what is this hidden region then?”.
The reason these are sometimes hard to answer is not simply because the answer is difficult but rather that the question was structured by a person or system that did not understand the answer that would be forthcoming. It is a sort of paradoxical problem and one that leads me to determine that question generation technology or “QGT” is one of the singular most important areas of AI research today. In this simple post I want to introduce the layman to the concept of “asking better questions”.
What do I mean by QGT? (here we go again!)...
Questions are often constructed (yes constructed) by a curious method of back ward chaining in which the poser has a relatively finite number of outcomes in mind. They then build the question as a result of known answers.
Imagine the poser of a question is perhaps looking to see if you might like to meet for lunch today. Imagine further that there are a number of types of food you both enjoy or have shared previously. A question is constructed :- “Where are you going for lunch today, Italian, Chinese or Turkish?”. The question seems innocent enough, however a lot is going on here. The construction of the question has imposed a limitation on the outcome of the answer that could not have existed without the poser knowing the answers already. In short, this is the kind of question that AI needs to sometimes avoid.
Asking the right question is the only way we can find new knowledge in big data. Once we have a well formed question or request, then and only then can we find new information that might have otherwise been missed.
In my restaurant example above, if the poser had included a fourth choice, lets say “Spanish”, the question would read : “Where are you going for lunch today, Italian, Chinese, Spanish or Turkish?”. Then this opens the receiver to new information that they did not know existed, in this case that there actually is another alternative to Chinese,Italian and Turkish food in their locality.
So how do we construct the right questions to find unknown unknowns?
What does that even mean? Most importantly for me it means looking at Big Data in reverse. Taking the data and actually developing questions that could be asked to arrive at a particular pattern, image, conclusion etc. In the following simple illustration I show all the numbers in a sequence by ascending order. 1,2,3,4, ,6,7,8,9,0 (note 5 is missing). Now assuming you haven't seen this sequence before, lets look at some hypothetical questions we might ask of a DL system that holds this knowledge.
- 1: What comes after 2?
- 2: does 9 precede 0?
- 3: how many 9's are there?
- 4: what is the average of the sequence 0-9?
These are all valid questions to which the system know the answer. They are easy. Some of the answers you get back may also be very wrong. In particular question 4. Because the system has no knowledge of 5, it will return the wrong answer to the question.
QGT, is about working backward from this lesson. It is about creating AI that is able to look at the sequence and generate a different question, the question : “What is between 4 and 6?”. That's not to say that the other questions are not useful, but they are effectively 'closed' in that no new knowledge can be created from asking them. Whereas the latter question can allow the DL system to create a new question, one that augments its current understanding of the data to allow new information to be gleaned.
Of course the example is very simple, but I hope the reader can extrapolate this up to larger scales or alternative uses. For example, if the data set was a set of stock levels of beverages (tea, coffee, lemonade …) then the system could have perhaps re worded the original question “Where are you going for lunch today, Italian, Chinese or Turkish?” to “Where are you going for lunch today, Italian, Chinese or Turkish? – bearing in mind these places have run out of coffee”.
Concluding..
The data we don't know is just as important as the data we do. Indeed, that famous saying “If you knew you knew nothing, you would know something – but you don't even know that” rings very true. AI developers need to focus on QGT as much as the relatively simple (if not huge) task of connecting dots to retrieve answers or Answer Retrieval Technology (ART).
