【Tyra Talk 2023】自然科學｜對抗落後者的去中心化差分隱私機器學習

機器學習需要大量的用戶資料來訓練模型，為了保有個別用戶的隱私，去中心化學習的技術因運而生。然而，某些裝置的運算速度可能較慢，拖累整體的訓練，造成所謂的「落後者問題」。林玄寅博士將分享如何擴展現有去中心化的差分隱私加強方法，以顧及整體的訓練時長，並分析其和正確率、隱私性之間的關係。
Speaker:
林玄寅(Hsuan-Yin Lin), Senior Research Scientist, Simula UiB, Norway
Hsuan-Yin Lin (Senior Member, IEEE) received a B.S. degree from the National Tsing-Hua University (NTHU), Taiwan, in 2007, and the M.S. and Ph.D. degrees from the National Chiao Tung University (NCTU), Taiwan, in 2008 and 2013, respectively. He was a Visiting Scholar at Universitat Pompeu Fabra, Barcelona, Spain, and TU Darmstadt, Germany. He is currently a Senior Research Scientist (since 2023) at Simula UiB, Norway. His current interests include privacy-preserving technologies, information-theoretic cryptography, coding in distributed storage systems, finite-length information theory, scheduling in millimeter-wave cellular networks, and distributed detection and estimation. In 2014, he was awarded the Honor Membership of the Phi Tau Phi Scholastic Honor Society of the Republic of China (Taiwan) and the New Partnership Program for the Connection to the Top Labs in the World (subsidized by the Ministry of Science and Technology, Taiwan). He was the Publicity Chair of the 9th International Workshop on Signal Design and its Applications in Communications, Dongguan, China. He is the chair of the IEEE Norway Section Information Theory Society Chapter and currently serving as a Guest Editor for Entropy: Special Issue “Information-Theoretic Privacy in Retrieval, Computing, and Learning” and a TPC member of the 2023 IEEE International Symposium on Information Theory.
研究領域 (Field):
自然科學、數學、統計、資訊通信科技
Abstract:
We consider the straggler problem in decentralized learning over a logical ring while preserving user data privacy. Especially, we extend the recently proposed framework of differential privacy (DP) amplification by decentralization by Cyffers and Bellet to include overall training latency--comprising both computation and communication latency. Analytical results on both the convergence speed and the DP level are derived for both a skipping scheme (which ignores the stragglers after a timeout) and a baseline scheme that waits for each node to finish before the training continues. A trade-off between overall training latency, accuracy, and privacy, parameterized by the timeout of the skipping scheme, is identified and empirically validated for logistic regression on a real-world dataset.
Chair:
陳建智 (Chien-Chih Chen), Ph.D. candidate at the University of Waterloo
-------------------
最即時資訊都在 project Tyra 的 FB、IG 及 LinkedIn 專頁，而線上演講也陸續上架到我們的 TH-cam 平台，歡迎追蹤及分享。若有任何問題或建議，歡迎來信 [e-mail] 與我們聯繫。我們期待與您一起促進海內外臺灣人學術圈間的交流及發展！
project Tyra 管理群
--
FB: / projecttyra
IG: / project_tyra_tw
LinkedIn: / projecttyra
TH-cam: www.youtube.co...