AI system is everywhere in the world. To make it more resilient, we need to investigate both the attack and the defense for it from the aspect of Software Engineering.