Hong-Yu Zhou

Hong-Yu Zhou
Postdoctoral fellow
Department of Biomedical Informatics
Harvard Medical School
Harvard University
Email: whuzhouhongyu at gmail.com

I am a postdoctoral fellow at the Harvard DBMI, working with Pranav Rajpurkar.
My research lies in the intersection of artificial intelligence, medicine, and data science.
I develop
Currently, I am actively researching the following pathways to enhance system scalability:

Acquisition of health-related insights at scales: How can one access a multitude of high-quality health insights (diagnoses, etc) at a notably reduced expense?
Scaling multimodal understanding: How can we leverage multimodal biomedical data (medical images, lab results, etc) to improve the modeling accuracy and to what extent can it do so?
Science of system scaling: Can we improve the modeling accuracy by increasing the data or model scale, for example, integrating large language models? How far can we go?

I finished my PhD at The University of Hong Kong, advised by Yizhou Yu and my master degree at Nanjing University, advised by Jianxin Wu. Before that, I received my undergraduate degree from Wuhan University.

Paper (Selected / Full)

2023

DDCoT: Duty-Distinct Chain-of-Thought Prompting for Multimodal Reasoning in Language Models.
Ge Zheng, Bin Yang, Jiajin Tang, Hong-Yu Zhou, Sibei Yang.
[Paper]
Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS), 2023.

Activate and Reject: Towards Safe Domain Generalization under Category Shift.
Chaoqi Chen, Luyao Tang, Leitian Tao, Hong-Yu Zhou, Yue Huang, Xiaoguang Han, Yizhou Yu.
[Paper]
International Conference on Computer Vision (ICCV), 2023.

A Transformer-based Representation-learning Model with Unified Processing of Multimodal Input for Clinical Diagnostics.
Hong-Yu Zhou*, Yizhou Yu*, Chengdi Wang*, Shu Zhang, Yuanxu Gao, Jia Pan, Jun Shao, Guangming Lu, Kang Zhang, Weimin Li.
[Paper, Code]
Nature Biomedical Engineering, 2023.

Advancing Radiograph Representation Learning with Masked Record Modeling
Hong-Yu Zhou*, Chenyu Lian*, Liansheng Wang, Yizhou Yu.
[Paper, Code]
International Conference on Learning Representations (ICLR), 2023.

Protein Representation Learning via Knowledge Enhanced Primary Structure Modeling
Hong-Yu Zhou*, Yunxiang Fu*, Zhicheng Zhang, Cheng Bian, Yizhou Yu.
[Paper, Code]
International Conference on Learning Representations (ICLR), 2023.

A Unified Visual Information Preservation Framework for Self-supervised Pre-training in Medical Image Analysis
Hong-Yu Zhou*, Chixiang Lu*, Chaoqi Chen, Sibei Yang, Yizhou Yu.
[Paper, Code]
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2023.

2022

Development and validation of an abnormality-derived deep-learning diagnostic system for major respiratory diseases
Chengdi Wang, Jiechao Ma, Shu Zhang, Jun Shao, Yanyan Wang, Hong-Yu Zhou, Lujia Song, Jie Zheng, Yizhou Yu, Weimin Li
[Paper, Code]
npj digital medicine, 2022.

ProCo: Prototype-aware Contrastive Learning for Long-tailed Medical Image Classification
Zhixiong Yang*, Junwen Pan*, Yanzhan Yang, Xiaozhou Shi, Hong-Yu Zhou, Zhicheng Zhang, Cheng Bian
[Paper, Code]
International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2022.

Relation Matters: Foreground-aware Graph-based Relational Reasoning for Domain Adaptive Object Detection
Chaoqi Chen, Jiongcheng Li, Hong-Yu Zhou, Xiaoguang Han, Yue Huang, Xinghao Ding, Yizhou Yu
[Paper, Code]
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2022.

GraVIS: Grouping Augmented Views from Independent Sources for Dermatology Analysis
Hong-Yu Zhou*, Chixiang Lu*, Liansheng Wang, Yizhou Yu
[Paper, Code]
IEEE Transactions on Medical Imaging (TMI), 2022.

MASS: Modality-collaborative semi-supervised segmentation by exploiting cross-modal consistency from unpaired CT and MRI images
Xiaoyu Chen*, Hong-Yu Zhou*, FengLiu, Jiansen Guo, Liansheng Wang, Yizhou Yu
[Paper, Code]
Medical Image Analysis, 2022.

Attribute Surrogates Learning and Spectral Tokens Pooling in Transformers for Few-shot Learning
Yangji He, Weihan Liang, Dongyang Zhao, Hong-Yu Zhou, Weifeng Ge, Yizhou Yu, Wenqiang Zhang
[Paper, Code]
IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR), 2022.

Advancing 3D Medical Image Analysis with Variable Dimension Transform based Supervised 3D Pre-training
Shu Zhang, Zihao Li, Hong-Yu Zhou, Jiechao Ma, Yizhou Yu.
[Paper, Code]
Technical Report.

Generalized Radiograph Representation Learning via Cross-supervision between Images and Free-text Radiology Reports
Hong-Yu Zhou*, Xiaoyu Chen*, Yinghao Zhang*, Ruibang Luo, Liansheng Wang, Yizhou Yu.
[Nature MI, arXiv, Code]
Nature Machine Intelligence, 2022.

2021

Preservational Learning Improves Self-supervised Medical Image Models by Reconstructing Diverse Contexts
Hong-Yu Zhou*, Chixiang Lu*, Sibei Yang, Xiaoguang Han, Yizhou Yu.
[Paper, Code]
International Conference on Computer Vision (ICCV), 2021.

nnFormer: Interleaved Transformer for Volumetric Segmentation
Hong-Yu Zhou*, Jiansen Guo*, Yinghao Zhang*, Lequan Yu, Liansheng Wang, Yizhou Yu.
[Paper, Code]
Technical Report.

ConvNets vs. Transformers: Whose Visual Representations are More Transferable?
Hong-Yu Zhou, Chixiang Lu, Sibei Yang, Yizhou Yu.
[Paper]
ICCV workshop on Multi-Task Learning in Computer Vision (DeepMTL).

Learning to Discover Multi-Class Attentional Regions for Multi-Label Image Recognition
Bin-Bin Gao, Hong-Yu Zhou.
[Paper, Code]
IEEE Transactions on Image Processing (TIP), 2021.

Generalized Organ Segmentation by Imitating One-Shot Reasoning Using Anatomical Correlation
Hong-Yu Zhou*, Hualuo Liu*, Shilei Cao*, Dong Wei, Chixiang Lu, Yizhou Yu, Kai Ma, Yefeng Zheng.
[Paper]
International Conference on Information Processing in Medical Imaging (IPMI), 2021.

SSMD: Semi-Supervised Medical Image Detection with Adaptive Consistency and Heterogeneous Perturbation
Hong-Yu Zhou*, Chengdi Wang*, Haofeng Li*, Gang Wang, Shu Zhang, Weimin Li, Yizhou Yu.
[Paper]
Medical Image Analysis, 2021.

Evaluation and Comparison of Accurate Automated Spinal Curvature Estimation Algorithms with Spinal Anterior-posterior X-Ray Images: The AASCE2019 Challenge
Liansheng Wang, Cong Xie, Yi Lin, Hong-Yu Zhou, Kailin Chen, Dalong Cheng, Florian Dubost, Benjamin Collery, Bidur Khanal, Bishesh Khanal, Rong Tao, Shangliang Xu, Upasana Upadhyay Bharadwaj, Zhusi Zhong, Jie Li, Shuxin Wang, Shuo Li.
[Paper]
Medical Image Analysis, 2021.

Recist-Net: Lesion Detection via Grouping Keypoints on Recist-Based Annotation
Cong Xie, Shilei Cao, Dong Wei, Hongyu Zhou, Kai Ma, Xianli Zhang, Buyue Qian, Liansheng Wang, Yefeng Zheng.
[Paper]
IEEE 18th International Symposium on Biomedical Imaging (ISBI), 2021.

MixSearch: Searching for Domain Generalized Medical Image Segmentation Architectures
Luyan Liu*^,†, Zhiwei Wen*, Songwei Liu*, Hong-Yu Zhou*^,†, Hongwei Zhu, Weicheng Xie, Linlin Shen, Kai Ma, Yefeng Zheng.
[Paper, Code]
Technical Report.

Bottom-Up Shift and Reasoning for Referring Image Segmentation
Sibei Yang, Meng Xia, Guanbin Li, Hong-Yu Zhou, Yizhou Yu.
[Paper, Code]
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.

2020

A Macro-Micro Weakly-supervised Framework for AS-OCT Tissue Segmentation
Munan Ning, Cheng Bian, Donghuan Lu, Hong-Yu Zhou, Shuang Yu, Chenglang Yuan, Yang Guo, Yaohua Wang, Kai Ma, Yefeng Zheng.
[Paper]
International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2020.

Difficulty-aware Glaucoma Classification with Multi-rater Consensus Modeling
Shuang Yu*, Hong-Yu Zhou*, Kai Ma, Cheng Bian, Chunyan Chu, Hanruo Liu, Yefeng Zheng.
[Paper]
International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2020.

Comparing to Learn: Surpassing ImageNet Pretraining on Radiographs by Comparing Image Representations
Hong-Yu Zhou*, Shuang Yu*, Kai Ma, Cheng Bian, Chunyan Chu, Hanruo Liu, Yefeng Zheng.
[Paper, Code]
International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2020.

Efficient and Effective Training of Covid-19 Classification Networks with Self-supervised Dual-track Learning to Rank
Yuexiang Li, Dong Wei, Jiawei Chen, Shilei Cao, Hongyu Zhou, Yanchun Zhu, Jianrong Wu, Lan Lan, Wenbo Sun, Tianyi Qian, Kai Ma, Haibo Xu, Yefeng Zheng.
[Paper]
IEEE Journal of Biomedical and Health Informatics (JBHI), 2020.

Learning Expectation of Label Distribution for Facial Age and Attractiveness Estimation
Bin-Bin Gao, Xin-Xin Liu, Hong-Yu Zhou, Jianxin Wu, Xin Geng.
[Paper]
Technical Report.

2019 and Before

Seg4Reg Networks for Automated Spinal Curvature Estimation
Yi Lin, Hong-Yu Zhou^†, Kai Ma, Xin Yang, Yefeng Zheng.
[Paper]
Ranked 1st place in 2019 MICCAI AASCE Challenge.
Computational Methods and Clinical Applications for Spine Imaging, 2019.

CodeAttention: Translating Source Code to Comments by Exploiting the Code Constructs
Wenhao Zheng, Hongyu Zhou, Ming Li, Jianxin Wu.
[Paper]
Frontiers of Computer Science, 2019.

When Semi-supervised Learning Meets Transfer Learning: Training Strategies, Models and Datasets
Hong-Yu Zhou, Avital Oliver, Jianxin Wu, Yefeng Zheng.
[Paper]
Technical Report.

ThiNet: Pruning CNN Filters for a Thinner Net
Jian-Hao Luo, Hao Zhang, Hong-Yu Zhou, Chen-Wei Xie, Jianxin Wu, Weiyao Lin.
[Paper, Code]
IEEE Transaction on Pattern Analysis and Machine Intelligence (TPAMI), 2018.

Age Estimation Using Expectation of Label Distribution Learning
Bin-Bin Gao, Hong-Yu Zhou, Jianxin Wu, Xin Geng.
[Paper, Code]
International Joint Conference on Artificial Intelligence (IJCAI), 2018.

Vortex Pooling: Improving Context Representation in Semantic Segmentation
Chen-Wei Xie, Hong-Yu Zhou, Jianxin Wu.
[Paper, Code]
Technical Report.

Sunrise or Sunset: Selective Comparison Learning for Subtle Attribute Recognition
Hong-Yu Zhou, Bin-Bin Gao, Jianxin Wu.
[Paper]
The British Machine Vision Association (BMVC), 2017.

Adaptive feeding: Achieving Fast and Accurate Detections by Adaptively Combining Object Detectors
Hong-Yu Zhou, Bin-Bin Gao, Jianxin Wu.
[Paper, Code]
International Conference on Computer Vision (ICCV), 2017.

A Transformer-based Representation-learning Model with Unified Processing of Multimodal Input for Clinical Diagnostics.
Hong-Yu Zhou*, Yizhou Yu*, Chengdi Wang*, Shu Zhang, Yuanxu Gao, Jia Pan, Jun Shao, Guangming Lu, Kang Zhang, Weimin Li.
A multimodal transformer for clinical diagnostics.
Nature Biomedical Engineering 2023 | Paper | Code

Advancing Radiograph Representation Learning with Masked Record Modeling.
Hong-Yu Zhou*, Chenyu Lian*, Liansheng Wang, Yizhou Yu.
Multimodal modeling for visual representation learning.
ICLR 2023 | Paper | Code

nnFormer: Volumetric Medical Image Segmentation via a 3D Transformer.
Hong-Yu Zhou*, Jiansen Guo*, Yinghao Zhang*, Xiaoguang Han, Lequan Yu, Liansheng Wang, Yizhou Yu.
A widely adopted baseline for 3D medical image segmentation.
TIP 2023 | Paper | Code

A Unified Visual Information Preservation Framework for Self-supervised Pre-training in Medical Image Analysis.
Hong-Yu Zhou*, Chixiang Lu*, Chaoqi Chen, Sibei Yang, Yizhou Yu.
TPAMI 2023 | Paper | Code

Generalized Radiograph Representation Learning via Cross-supervision between Images and Free-text Radiology Reports.
Hong-Yu Zhou*, Xiaoyu Chen*, Yinghao Zhang*, Ruibang Luo, Liansheng Wang, Yizhou Yu.
Nature Machine Intelligence 2023 | Paper | Code

Preservational Learning Improves Self-supervised Medical Image Models by Reconstructing Diverse Contexts.
Hong-Yu Zhou*, Chixiang Lu*, Sibei Yang, Xiaoguang Han, Yizhou Yu.
ICCV 2021 | Paper | Code

Comparing to Learn: Surpassing ImageNet Pretraining on Radiographs by Comparing Image Representations.
Hong-Yu Zhou*, Shuang Yu*, Kai Ma, Cheng Bian, Chunyan Chu, Hanruo Liu, Yefeng Zheng.
MICCAI 2020 | Paper | Code

Selected Awards

First place in AASCE 2019, held in conjunction with MICCAI 2019. We proposed an approach to accurately estimate the spinal curvature in Adolescent Idiopathic Scoliosis (AIS).
First place in Segthor 2019 Challenge, held in conjunction with ISBI 2019. We proposed a method to segment organs at risk segmentation in Computed Tomography (CT) images.
First place in AI4Health Challenge, held by Thales group which provides solutions for over 50% of worldwide radiology examinations. In this challenge, we proposed a novel methodology to accurately detect key points in X-rays with a mean error smaller than 0.3 pixels.
Second place in ODIR 2019, held by Peking University.

Academic Service

I regularly serve as reviewers for following conferences and journals.

Conference: for ICLR, ICML, NeurIPS, CVPR, ICCV, ECCV, AAAI, MICCAI, etc.
Journal: for Nature Machine Intelligence, TPAMI, TMI, MedIA, TKDE, TMM, TIP, Pattern Recognition, iScience (Cell family), and Scientific Reports.

Recent News