Learning by Doing: Building and Breaking a Machine Learning System -- Johann Rehberger | Summary and Q&A

1.4K views
November 6, 2020
by
Red Team Village
YouTube video player
Learning by Doing: Building and Breaking a Machine Learning System -- Johann Rehberger

TL;DR

In this talk, the speaker shares their machine learning journey, demonstrating the process of building, testing, and attacking a machine learning system.

Install to Summarize YouTube Videos and Get Transcripts

Questions & Answers

Q: What is the main objective of the speaker's talk?

The speaker's main objective is to share their machine learning journey, from building a system to testing and attacking it, in order to demonstrate the vulnerabilities and considerations in machine learning systems.

Q: How does the speaker suggest building resilience against adversarial attacks in machine learning systems?

The speaker suggests using adversarial training, where the system is trained with adversarial examples to improve its resilience. They also recommend implementing rate limiting, interpreting results differently, and continuously improving the model's accuracy to enhance the system's defense against attacks.

Q: What is the purpose of the image rescaling attack discussed by the speaker?

The purpose of the image rescaling attack is to demonstrate how resizing an image can fundamentally change its content, potentially leading to misclassification by the machine learning model. This highlights the need for caution when processing and resizing images in machine learning systems.

Q: How does the speaker propose mitigating the risk of backdooring in machine learning models?

The speaker suggests implementing hash validation to verify the integrity of the model file, as any modifications to the file can be detected. They also recommend conducting outside client checks to ensure consistent predictions and monitoring for repudiation attempts.

Summary & Key Takeaways

  • The speaker introduces their experience in offensive security and their journey into machine learning.

  • They discuss the resources they used to learn about machine learning and their participation in a machine learning security evasion competition.

  • The speaker presents their machine learning system, Husky Eye, which allows users to upload pictures of huskies and receives predictions on whether there is a husky in the image. They then explore various attacks on the system, including adversarial examples, image rescaling, and backdooring.

Share This Summary 📚

Summarize YouTube Videos and Get Video Transcripts with 1-Click

Download browser extensions on:

Summarize YouTube Videos and Get Video Transcripts with 1-Click

Download browser extensions on: