Accuracy and Robustness of Machine Learning Forecasts, Kenric P. Nelson
Machine learning algorithms are typically trained and tested based on classification or regression error. While the Kullback-Liebler or other information theoretic metrics may be utilized to assess the error in the probability forecasts, these metrics often measure relative performance without a clear sense of what constitutes an absolute standard of success. The interpretation of information theoretic metrics is clarified by translating them into a probability which can be compared with the classification metrics. Furthermore via the weighted generalized mean of predicted probabilities, which is a translation of the Tsallis and Renyi generalizations of entropy, the contrast between decisive and robust algorithms can be measured. Some illustrative examples show how Gaussian models lead to overconfident probability forecasts when complexity in the source of uncertainty actually involves distributions with slower decaying tails. Methods for improving the accuracy and robustness of machine learning models are discussed.
Dr. Kenric Nelson is a Senior Principal Engineer with Raytheon Integrated Defense Systems and Research Professor with Boston University Electrical & Computer Engineering. At Raytheon he leads projects on sensor management, tracking, discrimination, and debris mitigation. At Boston University he is developing a novel approach to information theory for complex systems. He has multiple inventions applying non-additive information theory to improve the robustness of radar processing and enable efficient probabilistic computation. His education in electrical engineering includes a B.S. degree Summa Cum Laude from Tulane University, a M.S. degree from Rensselaer Polytechnic Institute, and a Ph.D. degree from Boston University. His education in Program Management includes an Executive Certificate from MIT Sloan and certification with the Program Management Institute. His research interests include machine learning, complex adaptive signals and systems, and sensor systems.
I've spent the last several years working with Fortune 500 e-commerce teams on testing and optimization. The theme of our A/B testing program is often centered around deploying new technologies which rely on machine learning to provide some new competitive advantage.
We will discuss and show several examples of how online retailers and web applications can integrate machine learning in their production applications in order to generate a ROI. We will also walk through a basic technical implementation.
Kishan Supreet Alguri
Supreet is a final year PhD student from Department of Electrical and Computer Engineering, University of Utah working
with Dr. Joel B. Harley. His research interests include machine learning, transfer learning, complex wave propagation, and
Getting more from less with transfer learning:
Dictionary learning is a representation learning method which aims at finding a sparse representation of the input data (also known as sparse coding) in the form of a linear combination of basic elements as well as those basic elements them- selves. In this talk we discuss how dictionary learning has been used as a transfer learning method in solving problems of complex wave propagation in structures.
We are first going to briefly understand what compressive sensing and dictionary learning is, and then discuss the challenges of physical wave propagation and di iculties in obtaining wave propagation data. We then demonstrate how we used dictionary learning to solve wave field reconstruction challenges with very less experimental data by using the knowledge gained by learning from synthetic data.