Bradley J. Best, PhD

Rapid Tactics Generation

Simulation systems require extensive programming which makes their development prohibitively expensive. This project, completed for the US Office of Naval Research, focused on developing machine learning techniques for automatically generating computational models of behavior through task demonstration by human experts. Rather than programming flight maneuvers, we had real pilots execute tasks in a simulator and recorded their performance in detail (eye tracking, physiology monitoring, control inputs). Using these data, we developed techniques to learn to mimic their behavior (without programming it).

Our goal was to meet our customers' need for high-fidelity, low-cost virtual entities in simulation-based training and gaming markets To this end, I lead a team to develop novel machine learning algorithms and to develop a system to enable experts to demonstrate tasks in short vignettes.

The first challenge in this project was to distill the behavoiur of a subject matter expert into something a machine learning system could leverage. We approached this by using cognitive task analysis in concert with eye-tracking to decompose task performance into subtasks that machine learning could approach. These tasks needed to be relatively compact and well-constrained (even now, open-ended driving, for example, isn't realistic). This task decomposition then provided the "skeleton" upon which to hang individual machine learning models.

Given a task structure we obtained from working with a human expert, we developed a statistical machine learning algorithm specialized for these tasks. The algorithm needed to be abnle to learn from real-time streams and incorporate refining demonstrations provided by the expert (this was model development through demonstration, not programming!). The tables below show the performance of the algorithm we developed indexed against existing approaches.

The eventual system consisted of a set of user interfaces for use by non-programmers to define AI models for tactical game opponents. The interface would guide the expert in constructing a task model, grounded in their own vocvabulary, but connected to a simulation environment. Using these tools together, they could then demonstrate the various component tasks that the system would then learn how to perform.