Facial diagnosis became common due to discussions thousands of years ago of the link between disease and the face. Here we will use deep learning (CNN) techniques for the detection of facial diseases. In this article, we propose the use of computer-assisted facial diagnosis to perform deep transfer learning for facial recognition in several diseases. In our study, we performed computer-assisted facial diagnosis for several diseases Beta-thalassemia, hyperthyroidism, Down syndrome, and leprosy) using a limited data set. In the test, facial recognition deep transfer learning can achieve an overall accuracy rate of more than 90%, which is better than clinicians and traditional machine learning techniques.