Hashimoto's thyroiditis (HT) is the main cause of hypothyroidism. Here the authors develop a deep learning model for diagnosis of HT on a large multi-site dataset including image and video data. Hashimoto's thyroiditis (HT) is the main cause of hypothyroidism. We develop a deep learning model called HTNet for diagnosis of HT by training on 106,513 thyroid ultrasound images from 17,934 patients and test its performance on 5051 patients from 2 datasets of static images and 1 dataset of video data. HTNet achieves an area under the receiver operating curve (AUC) of 0.905 (95% CI: 0.894 to 0.915), 0.888 (0.836-0.939) and 0.895 (0.862-0.927). HTNet exceeds radiologists' performance on accuracy (83.2% versus 79.8%; binomial test, p < 0.001) and sensitivity (82.6% versus 68.1%; p < 0.001). By integrating serologic markers with imaging data, the performance of HTNet was significantly and marginally improved on the video (AUC, 0.949 versus 0.888; DeLong's test, p = 0.004) and static-image (AUC, 0.914 versus 0.901; p = 0.08) testing sets, respectively. HTNet may be helpful as a tool for the management of HT.