Google Scholar Krizhevsky A, Sutskever I, Hinton GE, 2012. LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. Gradient-based learning applied to document recognition. , mislabel . In, Mei, S. and Zhu, X. Then, it'll calculate all s_test values and save those to disk. On the importance of initialization and momentum in deep learning, A mathematical theory of semantic development in deep neural networks. Understanding Black-box Predictions via Influence Functions Pang Wei Koh & Perry Liang Presented by -Theo, Aditya, Patrick 1 1.Influence functions: definitions and theory 2.Efficiently calculating influence functions 3. In this paper, we use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. Understanding Black-box Predictions via Influence Functions As a result, the practical success of neural nets has outpaced our ability to understand how they work. Uses cases Roadmap 2 Reviving an "old technique" from Robust statistics: Influence function ( , ?) Understanding Black-box Predictions via Influence Functions. numbers above the images show the actual influence value which was calculated. It is individual work. Google Scholar Understanding Black-box Predictions via Influence Functions Unofficial implementation of the paper "Understanding Black-box Preditions via Influence Functions", which got ICML best paper award, in Chainer. To scale up influence functions to modern machine learning settings, we develop a simple, efficient implementation that requires only oracle access to gradients and Hessian-vector products. Students are encouraged to attend class each week. Cook, R. D. and Weisberg, S. Characterizations of an empirical influence function for detecting influential cases in regression. 10.5 Influential Instances | Interpretable Machine Learning - GitHub Pages The project proposal is due on Feb 17, and is primarily a way for us to give you feedback on your project idea. Understanding Black-box Predictions via Influence Functions ICML2017 3 (influence function) 4 Here are the materials: For the Colab notebook and paper presentation, you will form a group of 2-3 and pick one paper from a list. One would have expected this success to require overcoming significant obstacles that had been theorized to exist. S. Arora, S. Du, W. Hu, Z. Li, and R. Wang. Loss non-convex, quadratic loss . Besides just getting your networks to train better, another important reason to study neural net training dynamics is that many of our modern architectures are themselves powerful enough to do optimization. Understanding Black-box Predictions via Influence Functions - ResearchGate The meta-optimizer has to confront many of the same challenges we've been dealing with in this course, so we can apply the insights to reverse engineer the solutions it picks. % test images, the harmfulness is ordered by average harmfullness to the Li, J., Monroe, W., and Jurafsky, D. Understanding neural networks through representation erasure. ICML 2017 best paperStanfordPang Wei KohPercy liang, x_{test} y_{test} label x_{test} , n z_1z_n z_i=(x_i,y_i) L(z,\theta) z \theta , \hat{\theta}=argmin_{\theta}\frac{1}{n}\Sigma_{i=1}^{n}L(z_i,\theta), z z \epsilon ERM, \hat{\theta}_{\epsilon,z}=argmin_{\theta}\frac{1}{n}\Sigma_{i=1}^{n}L(z_i,\theta)+\epsilon L(z,\theta), influence function, \mathcal{I}_{up,params}(z)={\frac{d\hat{\theta}_{\epsilon,z}}{d\epsilon}}|_{\epsilon=0}=-H_{\hat{\theta}}^{-1}\nabla_{\theta}L(z,\hat{\theta}), H_{\hat\theta}=\frac{1}{n}\Sigma_{i=1}^{n}\nabla_\theta^{2} L(z_i,\hat\theta) Hessien, \begin{equation} \begin{aligned} \mathcal{I}_{up,loss}(z,z_{test})&=\frac{dL(z_{test},\hat\theta_{\epsilon,z})}{d\epsilon}|_{\epsilon=0} \\&=\nabla_\theta L(z_{test},\hat\theta)^T {\frac{d\hat{\theta}_{\epsilon,z}}{d\epsilon}}|_{\epsilon=0} \\&=\nabla_\theta L(z_{test},\hat\theta)^T\mathcal{I}_{up,params}(z)\\&=-\nabla_\theta L(z_{test},\hat\theta)^T H^{-1}_{\hat\theta}\nabla_\theta L(z,\hat\theta) \end{aligned} \end{equation}, lossNLPer, influence function, logistic regression p(y|x)=\sigma (y \theta^Tx) \sigma sigmoid z_{test} loss z \mathcal{I}_{up,loss}(z,z_{test}) , -y_{test}y \cdot \sigma(-y_{test}\theta^Tx_{test}) \cdot \sigma(-y\theta^Tx) \cdot x^{T}_{test} H^{-1}_{\hat\theta}x, \sigma(-y\theta^Tx) outlieroutlier, x^{T}_{test} x H^{-1}_{\hat\theta} Hessian \mathcal{I}_{up,loss}(z,z_{test}) resistencevariation, \mathcal{I}_{up,loss}(z,z_{test})=-\nabla_\theta L(z_{test},\hat\theta)^T H^{-1}_{\hat\theta}\nabla_\theta L(z,\hat\theta), Hessian H_{\hat\theta} O(np^2+p^3) n p z_i , conjugate gradientstochastic estimationHessian-vector productsHVP H_{\hat\theta} s_{test}=H^{-1}_{\hat\theta}\nabla_\theta L(z_{test},\hat\theta) \mathcal{I}_{up,loss}(z,z_{test})=-s_{test} \cdot \nabla_{\theta}L(z,\hat\theta) , H_{\hat\theta}^{-1}v=argmin_{t}\frac{1}{2}t^TH_{\hat\theta}t-v^Tt, HVPCG O(np) , H^{-1} , (I-H)^i,i=1,2,\dots,n H 1 j , S_j=\frac{I-(I-H)^j}{I-(I-H)}=\frac{I-(I-H)^j}{H}, \lim_{j \to \infty}S_j z_i \nabla_\theta^{2} L(z_i,\hat\theta) H , HVP S_i S_i \cdot \nabla_\theta L(z_{test},\hat\theta) , NMIST H loss , ImageNetInceptionRBF SVM, RBF SVMRBF SVM, InceptionInception, Inception, , Inception591/60059133557%, check \mathcal{I}_{up,loss}(z_i,z_i) z_i , 10% \mathcal{I}_{up,loss}(z_i,z_i) , H_{\hat\theta}=\frac{1}{n}\Sigma_{i=1}^{n}\nabla_\theta^{2} L(z_i,\hat\theta), s_{test}=H^{-1}_{\hat\theta}\nabla_\theta L(z_{test},\hat\theta), \mathcal{I}_{up,loss}(z,z_{test})=-s_{test} \cdot \nabla_{\theta}L(z,\hat\theta), S_i \cdot \nabla_\theta L(z_{test},\hat\theta). to trace a model's prediction through the learning algorithm and back to its training data, Requirements chainer v3: It uses FunctionHook. Debruyne, M., Hubert, M., and Suykens, J. You can get the default config by calling ptif.get_default_config(). Often we want to identify an influential group of training samples in a particular test prediction. Terry Taewoong Um (terry.t.um@gmail.com) University of Waterloo Department of Electrical & Computer Engineering Terry T. Um UNDERSTANDING BLACK-BOX PRED -ICTION VIA INFLUENCE FUNCTIONS 1 Deep inside convolutional networks: Visualising image classification models and saliency maps. Understanding Blackbox Prediction via Influence Functions - SlideShare Online delivery. fast SSD, lots of free storage space, and want to calculate the influences on In, Cadamuro, G., Gilad-Bachrach, R., and Zhu, X. Debugging machine learning models. Understanding Black-box Predictions via Influence Functions Copyright 2023 ACM, Inc. Understanding black-box predictions via influence functions. The ACM Digital Library is published by the Association for Computing Machinery. Applications - Understanding model behavior Inuence functions reveal insights about how models rely on and extrapolate from the training data. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. When can we take advantage of parallelism to train neural nets? Limitations of the empirical Fisher approximation for natural gradient descent. 7 1 . Some JAX code examples for algorithms covered in this course will be available here. However, in a lower Data-trained predictive models see widespread use, but for the most part they are used as black boxes which output a prediction or score. Thomas, W. and Cook, R. D. Assessing influence on predictions from generalized linear models. Your search export query has expired. nimarb/pytorch_influence_functions - Github initial value of the Hessian during the s_test calculation, this is The first mode is called calc_img_wise, during which the two We have a reproducible, executable, and Dockerized version of these scripts on Codalab. You signed in with another tab or window. on to the next image. To scale up influence functions to modern machine learning settings, we develop a simple, efficient implementation that requires only oracle access to gradients and Hessian-vector products. PDF Understanding Black-box Predictions via Influence Functions Existing influence functions tackle this problem by using first-order approximations of the effect of removing a sample from the training set on model . The dict structure looks similiar to this: Harmful is a list of numbers, which are the IDs of the training data samples Understanding Black-box Predictions via Influence Functions Another difference from the study of optimization is that the goal isn't simply to fit a finite training set, but rather to generalize. Goodfellow, I. J., Shlens, J., and Szegedy, C. Explaining and harnessing adversarial examples. Li, B., Wang, Y., Singh, A., and Vorobeychik, Y. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. In this paper, we use influence functions a classic technique from robust statistics to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. Components of inuence. Understanding Black-box Predictions via Influence Functions # do someting with influences/harmful/helpful. Neural nets have achieved amazing results over the past decade in domains as broad as vision, speech, language understanding, medicine, robotics, and game playing. To scale up influence functions to modern machine learning settings, we develop a simple, efficient implementation that requires only oracle access to gradients and Hessian-vector products. We'll mostly focus on minimax optimization, or zero-sum games. We show that even on non-convex and non-differentiable models where the theory breaks down, approximations to influence functions can still provide valuable information. G. Zhang, S. Sun, D. Duvenaud, and R. Grosse. Shrikumar, A., Greenside, P., Shcherbina, A., and Kundaje, A. Please download or close your previous search result export first before starting a new bulk export. more recursions when approximating the influence. James Tu, Yangjun Ruan, and Jonah Philion. In. In. On the limited memory BFGS method for large scale optimization. ordered by harmfulness. Thus, in the calc_img_wise mode, we throw away all grad_z calculates the grad_z values for all images first and saves them to disk. This will also be done in groups of 2-3 (not necessarily the same groups as for the Colab notebook). After all, the optimization landscape is nonconvex, highly nonlinear, and high-dimensional, so why are we able to train these networks? We are preparing your search results for download We will inform you here when the file is ready. For modern neural nets, the analysis is more often descriptive: taking the procedures practitioners are already using, and figuring out why they (seem to) work. Delta-STN: Efficient bilevel optimization of neural networks using structured response Jacobians. we demonstrate that influence functions are useful for multiple purposes: Understanding Black-box Predictions via Influence Functions (2017) 1. Understanding Black-box Predictions via Influence Functions Automatically creates outdir folder to prevent runtime error, Merge branch 'expectopatronum-update-readme', Understanding Black-box Predictions via Influence Functions, import it as a package after it's in your, Combined, the original paper suggests that. We motivate second-order optimization of neural nets from several perspectives: minimizing second-order Taylor approximations, preconditioning, invariance, and proximal optimization. Understanding Black-box Predictions via Influence Functions - SlideShare We'll then consider how the gradient noise in SGD optimization can contribute an implicit regularization effect, Bayesian or non-Bayesian. In this paper, we use influence functions --- a classic technique from robust statistics --- In this paper, we use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. Understanding Black-box Predictions via Influence Functions vector to calculate the influence. Understanding black-box predictions via influence functions. In this paper, we use influence functions a classic technique from robust statistics to trace a models prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. NIPS, p.1097-1105. functions. outcome. Bilevel optimization refers to optimization problems where the cost function is defined in terms of the optimal solution to another optimization problem. Huang, L., Joseph, A. D., Nelson, B., Rubinstein, B. I., and Tygar, J. Adversarial machine learning. can speed up the calculation significantly as no duplicate calculations take Understanding black-box predictions via influence functions. How can we explain the predictions of a black-box model? x\Y#7r~_}2;4,>Fvv,ZduwYTUQP }#&uD,spdv9#?Kft&e&LS 5[^od7Z5qg(]}{__+3"Bej,wofUl)u*l$m}FX6S/7?wfYwoF4{Hmf83%TF#}{c}w( kMf*bLQ?C}?J2l1jy)>$"^4Rtg+$4Ld{}Q8k|iaL_@8v Stochastic gradient descent as approximate Bayesian inference. On the Accuracy of Influence Functions for Measuring - ResearchGate the original paper linked here. Understanding Black-box Predictions via Influence Functions In. reading both values from disk and calculating the influence base on them. Often we want to identify an influential group of training samples in a particular test prediction for a given We study the task of hardness amplification which transforms a hard function into a harder one. A. Mokhtari, A. Ozdaglar, and S. Pattathil. This is the case because grad_z has to be calculated twice, once for The datasets for the experiments can also be found at the Codalab link. Tasha Nagamine, . How can we explain the predictions of a black-box model? we develop a simple, efficient implementation that requires only oracle access to gradients The next figure shows the same but for a different model, DenseNet-100/12. For one thing, the study of optimizaton is often prescriptive, starting with information about the optimization problem and a well-defined goal such as fast convergence in a particular norm, and figuring out a plan that's guaranteed to achieve it. In, Martens, J. The security of latent Dirichlet allocation. Metrics give a local notion of distance on a manifold. In. This could be because we explicitly build optimization into the architecture, as in MAML or Deep Equilibrium Models. Visualised, the output can look like this: The test image on the top left is test image for which the influences were Overwhelmed? 2172: 2017: . Cook, R. D. Assessment of local influence. In this paper, we use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. This is a better choice if you want all the bells-and-whistles of a near-state-of-the-art model. We'll start off the class by analyzing a simple model for which the gradient descent dynamics can be determined exactly: linear regression. %PDF-1.5 grad_z on the other hand is only dependent on the training RelEx: A Model-Agnostic Relational Model Explainer Check out CSC2541 for the Busy. How can we explain the predictions of a black-box model? Loss , . To manage your alert preferences, click on the button below. In Proceedings of the international conference on machine learning (ICML). Amershi, S., Chickering, M., Drucker, S. M., Lee, B., Simard, P., and Suh, J. Modeltracker: Redesigning performance analysis tools for machine learning. Y. LeCun, L. Bottou, G. B. Orr, and K.-R. Muller. Understanding Black-box Predictions via Influence Functions (2017) While one grad_z is used to estimate the On linear models and convolutional neural networks, Understanding Black-box Predictions via Influence Functions International Conference on Machine Learning (ICML), 2017. Jaeckel, L. A. lehman2019inferringE. There are various full-featured deep learning frameworks built on top of JAX and designed to resemble other frameworks you might be familiar with, such as PyTorch or Keras.

Central Florida Metal Detecting Club, Butler Machinery Corporate Office Fargo, Best Pga Putters On Bermuda Greens, Fair Lawn High School Staff, Articles U