Guanhua Wang｜王冠华
Senior Researcher

DeepSpeed @ Microsoft

Email:      moc.tfosorcim@gnawauhnaug
BIO

Biography


I am a Senior Researcher in DeepSpeed at Microsoft.

I earned my Computer Science PhD from AMPLab / RISELab at UC Berkeley in 2022, advised by Prof. Ion Stoica. I received my M.Phil. degree in Computer Science from Hong Kong University of Science and Technology in 2015, advised by Prof. Lionel M. Ni. I got my B.Eng. degree in Computer Science from Southeast University, China, in 2012.

CV                     Google Scholar                     GitHub                     Photos


Research Interests


Distributed Machine Learning, Distributed Systems, Networking

NEWS

PUBLICATIONS

Conferences

  1. [ICLR'24] ZeRO++: Extremely Efficient Collective Communication for Giant Model Training

    Guanhua Wang, Heyang Qin, Sam Ade Jacobs, Xiaoxia Wu, Connor Holmes, Zhewei Yao, Samyam Rajbhandari, Olatunji Ruwase, Feng Yan, Lei Yang, Yuxiong He

    In Proceedings of The Twelfth International Conference on Learning Representations (ICLR)

  2. [MLSys'21] Wavelet: Efficient DNN Training with Tick-Tock Scheduling

    Guanhua Wang, Kehan Wang, Kenan Jiang, Xiangjun Li, Ion Stoica

    In Proceedings of Fourth Conference on Machine Learning and Systems (MLSys)

  3. [MLSys'21] sensAI: ConvNets Decomposition via Class Parallelism for Fast Inference on Live Data ( Website | Code )

    Guanhua Wang, Zhuang Liu, Brandon Hsieh, Siyuan Zhuang, Joseph Gonzalez, Trevor Darrell, Ion Stoica

    In Proceedings of Fourth Conference on Machine Learning and Systems (MLSys)

  4. [MLSys'20] Blink: Fast and Generic Collectives for Distributed ML (Slides)

    Guanhua Wang, Shivaram Venkataraman, Amar Phanishayee, Jorgen Thelin, Nikhil Devanur, Ion Stoica

    In Proceedings of Third Conference on Machine Learning and Systems (MLSys)

  5. [MobiCom'14] We Can Hear You with Wi-Fi!

    Guanhua Wang, Yongpan Zou, Zimu Zhou, Kaishun Wu, Lionel M. Ni

    In Proceedings of ACM MobiCom 2014

  6. [INFOCOM'14] SimCast: Efficient Video Delivery in MU-MIMO WLANs

    Guanhua Wang, Kaishun Wu, Qian Zhang, Lionel M. Ni

    In Proceedings of IEEE INFOCOM 2014

  7. [ICDCS'14] TiM: Fine-Grained Rate Adaptation in WLANs

    Guanhua Wang, Shanfeng Zhang, Kaishun Wu, Qian Zhang, Lionel M. Ni

    In Proceedings of IEEE ICDCS 2014

Journals

  1. [TMC] We Can Hear You with Wi-Fi!       (Featured Article)

    Guanhua Wang, Yongpan Zou, Zimu Zhou, Kaishun Wu, Lionel M. Ni

    IEEE Transactions on Mobile Computing, Vol. 15, No. 11, November 2016

  2. [TMC] TiM: Fine-Grained Rate Adaptation in WLANs

    Guanhua Wang, Shanfeng Zhang, Kaishun Wu, Qian Zhang, Lionel M. Ni

    IEEE Transactions on Mobile Computing, Vol. 15, No. 3, March 2016

  3. [TWC] CSMA/SF: Carrier Sense Multiple Access with Shortest First

    Guanhua Wang, Kaishun Wu, Lionel M. Ni

    IEEE Transactions on Wireless Communications, Vol. 13, No. 3, March 2014

WORK EXPERIENCE

  • Software Engineering Intern @ Oasis Labs, Berkeley, CA 05/2018 - 08/2018
  • Implement distributed key-value store for data query and data migration over blockchain.
  • Mentor: Dawn Song (Professor at EECS, UC Berkeley)

  • Research Intern @ Microsoft Research (MSR), Redmond, WA 05/2017 - 08/2017
  • Speeding up communication among GPUs within DGX-1 machine, and across multiple machines.
  • Mentor: Amar Phanishayee (Senior Principal Researcher)

  • Research Intern @ Microsoft Research (MSR), Redmond, WA 05/2016 - 08/2016
  • Improving image query accuracy (Microsoft Bing Image Search Engine) using CNTK Deep Learning Toolkits.
  • Mentors: Sanjeev Mehrotra (Principal Software Architect), Jin Li (Partner Researcher Manager)

HONORS AND AWARDS

  • 2021              Half Marathon Finisher @ San Francisco (2h15min)
  • 2019              Half Marathon Finisher @ Berkeley (2h02min)
  • 2017              Half Marathon Finisher @ San Francisco (2h17min)
  • 2012-2015     Postgraduate Studentship at HKUST
  • 2014              IEEE INFOCOM 2014 Student Travel Grant
  • 2014              HKUST Research Travel Grant
  • 2012              First Prize of the Outstanding Graduation Project in Jiangsu Province (Top 0.1%)
  • 2011-2012     IBM Chinese Excellent Student Scholarship
  • 2010-2011     China Aerospace Science and Technology Corporation (CASC) Scholarship
  • 2009-2010     Student in Free Enterprise (SIFE) Elite Student Certificate
  • 2008-2009     Champion of Southeast University Soccer Cup
  • 04-06/2012    Winner of Outstanding Graduation Project in Southeast University
  • 11-12/2010    Award of Excellence in Southeast University FPGA Design Competition
  • 09-10/2010    Award of Excellence in SIFE 2010 National Competition
  • 09-10/2010    Third Prize of National Undergraduate Mathematical Contest in Modeling (MCM)
  • 07-09/2010    Second Prize of National Undergraduate Electronic Design Contest
  • 05-07/2010    Top 10 Outstanding Team in National VMware Cloud Computing Contest
  • 04-05/2010    Award of Excellence in 2010 Southeast University Smart Car Competition
  • 03-04/2010    Third Prize of Southeast University Embedded System Design Competition
  • 02-04/2010    Second Prize of 2010 Microsoft Image Cup Programming Contest
  • 02-04/2010    Second Prize of Southeast University RoboCup Competition
  • 01-02/2010    Third Prize of Southeast University Electronic Design Competition

EXTRACURRICULAR ROLES

  • Captain Men’s Soccer Team in School of Computer Science and Engineering, Southeast University (2009-2012)
  • Core Member Southeast University SIFE Team (2009-2011)