Machine Learning Meetup
This weekly seminar series, hosted by the computer vision research team at FeatureX, is open to students and professionals who share an interest in gathering with like-minded machine learning researchers. This series focuses on current and influential papers in machine learning, and brings active participants together around one relevant paper each week. The presenter will introduce the background of the paper and review the findings. Attendees are expected to have read the paper and be ready to participate in group discussions about the research content and its implications.
Space is limited and RSVP’s are mandatory for this event. Please email Emily Rogers at firstname.lastname@example.org if you plan to attend. If your plans change, please update us so we can offer space to someone else.
Paper: Do CIFAR-10 Classifiers Generalize to CIFAR-10? by Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, Vaishaal Shankar
Abstract: Machine learning is currently dominated by largely experimental work focused on improvements in a few key tasks. However, the impressive accuracy numbers of the best performing models are questionable because the same test sets have been used to select these models for multiple years now. To understand the danger of overfitting, we measure the accuracy of CIFAR-10 classifiers by creating a new test set of truly unseen images. Although we ensure that the new test set is as close to the original data distribution as possible, we find a large drop in accuracy (4% to 10%) for a broad range of deep learning models. Yet more recent models with higher original accuracy show a smaller drop and better overall performance, indicating that this drop is likely not due to overfitting based on adaptivity. Instead, we view our results as evidence that current accuracy numbers are brittle and susceptible to even minute natural variations in the data distribution.