1 Introduction

Insulin is a hormone that controls sugar levels in the body. Diabetes is a non-reversible disorder that arises when the body is unable to effectively utilize the insulin produced by the pancreas or fails to produce sufficient insulin. A lack of insulin results in high blood sugar levels, the primary cause of diabetes. Currently, diabetes is among the most prevalent diseases affecting people of all ages, particularly the elderly. Approximately 11% of the US population has diabetes. Worldwide, 537 million adults have diabetes. This number is projected to rise to nearly 600 million by 2030 and 783 million by 2045. High blood sugar damages the vessels in the heart, eye, kidney, and so on [1].

Figure 1 illustrates how insulin imbalance leads to an increased risk of diabetes. Increasing sugar levels can cause various disorders, including obesity; kidney, heart, and eye diseases; polycystic ovary syndrome (PCOS); and cancer. Among these, this study specifically focuses on eye-related complications. Diabetic retinopathy (DR) is one such complication resulting from diabetes. It occurs due to damage to the blood vessels in the retina, the light-sensitive tissue at the back of the eye [2]. DR is one of the leading causes of blindness, predominantly affecting individuals aged between 25 and 74 years with diabetes mellitus. Early medical intervention can prevent blindness in more than 90% of cases, highlighting the critical need for early detection and timely treatment [3]. Initially, diabetic retinopathy may present with no symptoms or only mild vision impairment, but without intervention, it can progress to complete vision loss. Despite the serious consequences of DR, many patients fail to associate their vision problems with high blood sugar levels, often attributing them to the normal aging process. Detecting DR in the early stages is crucial for preventing irreversible damage.

Fig. 1
Fig. 1
Full size image

Increase of sugar in blood due to insulin resistance: This figure shows the cycle of high blood sugar that accompanies insulin resistance. The cycle starts when insulin is produced in response to the intake of sugar, followed by cellular resistance to insulin, resulting in higher blood sugar levels. This causes an increase in hunger, subsequent overeating, and extra sugar being stored as fat, continuing the cycle

Deep Learning (DL) has had a transformative impact on medical science, particularly in disease detection. DL algorithms offer powerful tools for identifying diseases such as cancer, tumors, and diabetic retinopathy. In the medical sector, DL techniques focus on developing algorithms capable of diagnosing disorders accurately by analyzing clinical data [4]. Yet, DR remains one of the most underdiagnosed eye diseases, with many unaware that diabetes can lead to significant vision impairment or blindness.

Although prior research has explored DR detection, many studies suffer from limitations such as low model accuracy, insufficient data availability [5], poor data visualization, and lack of interpretability. Moreover, many systems remain reliant on traditional human grading [6], neglecting recent advancements in automated analysis. These shortcomings highlight a clear research gap and the need for more accurate, efficient, and scalable solutions for DR detection.

Addressing these limitations, we introduce DR-RetinaNet, a model based on a customized DenseNet201 backbone. By leveraging pre-trained networks and innovative preprocessing strategies, DR-RetinaNet aims to offer a more robust and clinically applicable solution for DR detection.

The key contributions of this study include:

  • We propose DR-RetinaNet, a deep learning-based model for diabetic retinopathy detection, designed as an efficient system for early and accurate disease screening.

  • To optimize the performance of DR-RetinaNet, we have preprocessed the dataset through normalization and data balancing techniques, ensuring its suitability for the proposed model.

  • The study incorporates a comprehensive comparison of performance metrics, including accuracy, precision, and recall, between DR-RetinaNet and state-of-the-art models.

The remainder of this paper is organized as follows. A summary of earlier studies on various eye conditions is provided in Sect. 2. In Sect. 3, we discuss our proposed methodology, including preprocessing techniques. In the following sections, we focus on the evaluation matrix and the performance analysis. Section 6 provides the discussion and final conclusions of this study.

2 Literature Review

In this study, we surveyed various types of papers to analyze past research. While our primary focus is diabetic retinopathy (DR), we deliberately included studies on other ocular conditions such as thyroid eye disease, retinoblastoma, and fatigue-related ocular disorders. The motivation for this broader coverage is twofold. First, it highlights the rapid and diverse applications of deep learning across the eye disease domain, showing how AI-driven solutions are being explored in multiple clinical contexts. Second, by situating DR within this wider landscape, we underscore why diabetic retinopathy deserves particular attention: it is both one of the most prevalent and one of the most underdiagnosed vision-threatening conditions. Thus, the broader survey not only provides background on the state of eye disease research but also reinforces the urgency of advancing automated methods for DR detection.

In order to automatically identify and classify DR fundus pictures according to their severity, Fayyaz et al. [6] employed feature extraction with an SVM algorithm. The data augmentation plan comprised rotations of \(90^{\circ }\) and \(180^{\circ }\) and vertical and horizontal flips. AlexNet and Resnet101 were used to obtain the features. The 750-feature experiment demonstrates that the proposed method had a 93% accuracy rate. The dataset utilized in this study was thoroughly labelled by a physician. The dataset was human-interpreted which could have caused errors. In [7], the authors created and verified DeepDR Plus, a system with deep learning that uses only fundus pictures to forecast the time to progress with DR over five years. The fundus model’s ability to predict outcomes was assessed by dividing diabetic patients’ eyes into two groups. Low and high-risk categories were identified using the developmental dataset’s mean risk score, which was determined using fundus models. Initially, the authors pretrained the algorithm using 717,308 fundus pictures from 179,327 patients with diabetics. Subsequently, they used a multiethnic dataset containing 118,868 photos from 29,868 diabetic subjects to train and evaluate the system. To assess the period of DR development, the technique produced a C index of 0.754–0.846 along with Brier values of 0.153–0.241 for every period for up to five years. Additionally, real-world cohorts of individuals with diabetes were used to validate the method. The authors acknowledge several limitations of their study. They claimed that only the Chinese population was used to train their DeepDR Plus system. More training on a larger range of demographic and clinical datasets may enhance the prediction performance and make it more applicable to other groups. Additionally, individuals with varying treatment regimens might demonstrate varied fundus model performance; further research is necessary to clarify this issue. In research work [8], several deep-learning applications and imaging methods for DR were showcased. They discussed different methods using different types of datasets. The authors stated that a Convolutional Neural Network (CNN) can be used with the MESSIDOR-2 fundus image dataset to identify DR; however, the limitation is that a large dataset will not be considered. If the Optical Coherence Tomography (OCT) dataset uses a three-layered CNN, then the huge challenge is the use of OCT images alone in the implementation. The accuracy of diagnosis has greatly improved with the application of deep learning algorithms in medical imaging systems. To identify diabetic retinopathy, conventional research has focused on deep-layered architectures and CNN models. The scarcity of publicly accessible datasets makes DR diagnosis difficult to achieve. Though they still have trouble detecting the damaged lesions, new developments in DL have shown encouraging classification findings. To identify diabetic retinopathy, conventional research has focused on deep-layered architectures and CNN models.

Retinopathy caused by diabetes is a condition that rapidly impairs eyesight. Preprocessing methods were used to input the color retinal pictures. The quality of the photos was enhanced, using these preprocessing methods. The features were removed from the previously processed pictures to begin the categorization process. This study succeeded in classifying DR using the PNN, Bayes theory, and SVM into two categories: NPDR and PDR. All three classification approaches performed well; however, the findings show that SVM is more effective than PNN and Bayes Theory. As a result, this effort has produced an effective Diabetic Retinopathy Diagnosis technique that assists in early disease diagnosis and minimizes manual labor. In addition, 130 photographs were used to operate this system. However, by extracting better features, adding more data to each class, and merging their model with other pattern classification algorithms, the authors may increase the right categorization accuracy [9]. Researchers have demonstrated the effectiveness of computerized DR screening owing to recent advancements in deep learning technology. In this work, the researchers used deep learning and ultra-wide-field fundus photography to build a diabetic retinopathy diagnosis method [10]. A detailed review of automated diabetic retinopathy detection techniques has been presented in a survey publication. The evaluation was divided into five categories: accuracy, sensitivity, specificity, databases, and methodologies. Along with the public databases, the databases used were Messidor, Kaggle, DIARETDB1, and E-Ophtha. This offers more details on the current databases and a potential way for researchers to identify proficiencies that are accessible for the diagnosis of diabetic retinopathy. Growing databases provide future research projects with a solid foundation. Initial detection of diabetic retinopathy is a useful follow-up procedure. In addition, the efficiency of the algorithm must continue to improve. It may be beneficial for future research to investigate the origin of blood vessels [11]. An essential part of optical diagnostics is the identification of various eye disorders using fundus images. This paper presents the Fundus-DeepNet system, an automated multilabel DL system. A thorough preprocessing process of the images is started by the research, which includes data augmentation, noise reduction, contrast improvement, enlarging the picture, and circular border cropping. Then, utilising several deep learning blocks as feature descriptors, including the Attention Block and the High-Resolution Network (HRNet), discriminative deep feature representations are recovered. However, prospective research and clinical settings may be used to deploy it [12]. By gathering DR data from numerous sources and classifying it based on the expertise of clinical specialists, another investigation was started. The labeled dataset was used to train and validate the model, which produced 89.45% specificity, 97.42% sensitivity, and 93.40% accuracy. The model was put into practice and patients were assessed in real time at SIOVS. The highest-quality photographs were guaranteed by the creative technique. The images are classified as either DR-positive or DR-negative by the model. Clinical professionals assessed the final images to gauge the model’s performance in real time. Findings Clinical experts reported that the model’s accuracy, sensitivity, and specificity were 93.30%, 97.30%, and 92.90%, respectively [5].

A survey paper published in 2023, thoroughly identified which deep learning-based classification is suitable for diabetic retinopathy detection. The studies in this area were divided into two groups: those that relied on DR detection and those that relied on DR severity assessment. Fundus pictures were categorized into severity levels in most studies. The suggested techniques for assessing diabetic retinopathy rely on deep learning with various CNN architectures. In these studies, transfer learning was used extensively for this purpose. This is because the established models performed well in picture categorization tasks. This covers deep-learning models and architectures. Furthermore, preprocessing methods have been applied in many experiments to enhance the performance. Preprocessing methods that are frequently used include green channel extraction, CLAHE, scaling, and grayscale conversion. These methods reduce extra noise from the photos, which helps to improve the feature extraction process [13]. 33 studies based on DR detection were examined in a different survey study published in 2020. All the deep learning techniques used in this study were used to alter the DR screening system. To enhance the number of images and avoid overfitting during the training phase, the majority of studies employed data augmentation. There was virtually little difference in the percentage of researchers who chose to employ pre-existing frameworks with transfer learning and those who constructed their own CNN structure. It takes a lot of time and effort to build a new CNN architecture from scratch, but transfer learning accelerates and simplifies the process. To make a knowledgeable choice between the two trends, the authors advised researchers to concentrate on this issue and conduct more research [14]. In 2023, a different study examined the features of diabetic retinopathy. They take a long time, are prone to inaccuracy, and are challenging to examine and diagnose. Fundus images from many datasets were combined, and two modules—exudates and microaneurysms, were graded to determine the severity of diabetic retinopathy. Various image processing methods have been used to analyze the features of diabetic retinopathy. Based on fundus images, various algorithms have been used to train, assess, and forecast the severity of diabetic retinopathy. While support vector machines and k nearest neighbors execute effectively with higher accuracy for grading using exudates, decision trees perform well with an accuracy of 99.9% for grading using microaneurysms [15].

In this study, the researchers not only focused on the DR detection problem but also, investigated various types of detection methods for eye disorders. Thyroid eye disease (TED), has now become one of the vital disorders related to the eye. TED is an autoimmune disease that can cause significant morbidity unless diagnosed and treated in time. The AI model used in this study provides a quantitative predictive tool that can distinguish the TED state from the normal state. This could potentially support the screening of TED by frontline health workers, stratifying referrals to endocrinologists and oculoplastic surgeons in the process of further development and validation. Owing to these limitations in current literature, the model cannot be scaled for wider use and generalized to patients in the United States [16]. Retinoblastoma is a common malignant intraocular tumor in children. When the illness progresses at presentation, the prognosis for eyesight, globe function, and survival is poor. The illness that progresses locally is mainly caused by a delayed diagnosis and appearance. Here, their goal was to develop an AI model that could be applied to fundus images captured using a low-cost nonmydriatic fundus camera. Using this approach, children may be screened across the community to detect RB early, before symptoms of strabismus or leukocoria appear. Based on a successful Mobile-Net model, the transfer learning implementation eliminated the final layer using an integrated optic disc and tumor detection. Using the OpenCV technique in conjunction with DL, the AI model could identify RB with a sensitivity of 96% and specificity of 94%. This research has several drawbacks, such as a comparatively small sample size and uneven distribution of RB among the groups according to the ICIoR. This study aimed to train the RB classifier to evaluate fundus pictures, therefore anterior segment variables that are predictive of the RB group were not included at this time. Furthermore, they did not evaluate their model using fundus photographs taken with various cameras or with fundus photos of different races. Therefore, it is uncertain whether this AI model can be applied to the aforementioned situations [17].

One helpful diagnostic method for assessing the posterior part of the eye is OCT imaging. The aforementioned condition has a substantial impact on the specificity of diagnosis, monitoring of various physiological and pathological processes, response to therapy, and evaluation of therapeutic efficacy in a number of clinical practice domains. Therefore, accurate diagnosis, automated image analysis, and classification are essential. To categorize retinal OCT, the authors of this work suggested an improved optical coherence tomography model. The random forest and modified ResNet50 algorithms, which are included in the training strategy of the suggested study to improve the performance, are the foundation of this model. Examination of choroidal characteristics with increased depth imaging on OCT images was not included [18]. The process of applying the knowledge gained from solving one problem to solving another is known as transfer learning. It seeks to provide computer systems with the capacity to learn from past experiences and knowledge. Transfer learning has various substantial advantages over traditional machine learning methods in terms of the transfer of learned knowledge. This study collected 1980 eye contour images of 96 different individuals to solve the problem of person recognition from eye shots. The person, age, and sex categories were applied to the data gathered. The Random Forest technique for person estimation was used for classification in the eye identification task after 32 distinct transfer learning algorithms in Python were used to extract features. The study’s findings indicate that 30 distinct classification algorithms—ResNet50 were the most effective, and the data were also categorized based on the age and gender of the participants. In the future, eye pictures from more people can be used for categorization investigations. Research may be conducted using eye pictures of both healthy and ill subjects to identify individuals with chronic conditions including diabetes, hypertension, and other illnesses [19].

A vast area of research is a must. In the first step, the authors thoroughly investigated the papers related solely to the detection of diabetic retinopathy. After that, they broadened their area and investigated eye disorders, but this time they focused on other diseases to get the idea of recent research regarding eye disease. In this phase, researchers expanded their study. In this phase, they investigated different types of detection techniques, related to eye and image detection. Currently, human identification and authenticity are the most crucial elements. External identity sources, such as cards of identity with RFID enabled and magnetic interface cards, are sources of malfunction. The internal sources of human identity can be identified by observing visible human components, including the iris, sclera, lip shape, nose shape, ear shape, palm, and face. Human information samples, such as deoxyribonucleic acid and body smells, were employed for identification using the forensic-based method. The iris, retina, and sclera are biometric sources for the human eye. This study suggests a strategy to feed precise iris boundaries that have already been preprocessed into an AlexNet deep learning neural network-based classification system. The pupil center and boundary were first recorded and determined using the eye pictures provided. In previously published efforts, Alexnet was fed raw eye pictures for feature extraction and classification rather than preprocessing. Iris characteristics and iris occultation, such as eye lazes and lids, are the extracted features of the current system. The accuracy of the current method was between 95.63% and 98.6%. In the future, a cross-compiler mode will be implemented to perform the suggested task. Adequate hardware is necessary for real-time implementation of the proposed system [20]. One study addressed the development of an all-encompassing drowsiness detection system that predicts an operator’s ocular state to identify their level of tiredness and alerts them ahead of time of any significant risks to road safety. The authors of this paper proposed a CNN model for eye state categorization using state-of-the-art DL and digital image processing techniques, and tested it with three CNN models. A new CNN model called the 4D model was created to evaluate fatigue according to eye conditions. The MRL Eye dataset, which included 47,173 pictures of a single eye in both open and closed conditions, was used to build the model. A class activation map was used to visualize model learning. The input stream from the camera was processed using the OpenCV library during the test. According to the MRL Eye dataset, the network accuracy on the ROI images was 97.53%, 95.03%, and 95.93% for the 4D model, VGG16, and VGG19, respectively. Head posture can also be used as an indicator of sleepiness. There may be substantial differences between the driving behavior-based measures used in the simulations and actual driving scenarios [21]. A comprehensive study on using smartphone photography for oral disease diagnosis has not been conducted yet. The authors developed a DL-based method for automated oral disease detection using smartphone images, implementing a centered rule method for high-quality image capture and a resampling technique to address variability from handheld cameras. They created a medium-sized oral dataset with five disease types and evaluated a pretrained HRNet model, demonstrating significant improvement in smartphone-based early cancer diagnosis capabilities [22].

In this paper, a method of forecasting the disease based on a patient’s age, gender, and symptoms was described. The Weighted KNN model predicted diseases with the highest accuracy of 93.5%. Nearly every machine learning model produced accurate results. Certain models have very low accuracy and are unable to forecast the disease because they were parameter-dependent. This model would speed up healing while also assisting in reducing the amount of money needed to treat the illness [23].

In [24], the authors focused on the performance of Vision Transformers (ViTs) in classifying brain tumors based on T1-weighted MRI images. In this respect, the authors used 3064 T1-weighted contrast-enhanced MRI slices of 233 patients diagnosed with meningiomas, gliomas, and pituitary tumors. This will involve fine-tuning pre-trained ViT models and testing their individual and ensembled performances. The results of this study show that the ViT ensemble, for the classification of brain tumors at 384\(\times\)384 resolution, achieved an accuracy of 98.7%, outperforming traditional CNN models. The study points out some limitations, such as the computational demands of training ViTs at higher resolutions, which require substantial resources. The reference paper [25], focused on the application of ViTs to traffic sign classification, bench-marking them against CNNs. The investigation has used three publicly open datasets on traffic signs, namely GTSRB, the Indian Traffic Sign Dataset, and the Chinese Traffic Sign Dataset. They experimented with five variants of ViT and seven CNN architectures, showing that while ViTs perform well on smaller datasets, they do not perform as well on larger datasets compared to CNNs. This study identified that CNNs outperform ViTs by up to 12.81%, 2.01%, and 4.37% for the German, Indian, and Chinese traffic sign datasets, respectively. The authors also give some suggestions for improving the performance of ViTs: better pre-training techniques and the development of new hybrid models. In [26], the authors proposed a modality fusion vision transformer for collaborative classification in hyperspectral image-LiDAR data and has put forward a new way to fuse such heterogeneous features. The authors designed a novel multimodal cross-attention mechanism for effective fusion that avoids issues of feature misalignment often seen with existing methods. The MFViT model leverages spectral self-attention to preserve spatial features in an image while leveraging spectral information from HSI. Experiments show that the model performs well compared to state-of-the-art methods and delivers high accuracy for classification in popular benchmark datasets such as Trento, Houston 2013, and Houston 2018. However, high computational complexity for the model has been mentioned, which can be optimized further.

Table 1 Summary of literature review on diabetic retinopathy detection: the table captures an overview of the Diabetic Retinopathy (DR) detection literature review in the context of datasets used, methods applied, and their respective limitations
Full size table

Drawing on prior studies, our work directly addresses several recurring limitations. Instead of relying on hand-crafted features with SVMs or binary NPDR/PDR pipelines [9, 15], we adopt an end-to-end DenseNet201 backbone with a compact head for five-class DR classification, ensuring finer granularity. To overcome instability on limited and imbalanced datasets repeatedly noted in survey papers [8, 13, 14], we employ selective layer freezing and keep Batch Normalization in inference mode during fine-tuning. We also apply standardized preprocessing and simple but effective augmentations to mitigate heterogeneous image quality and resolution issues highlighted in the literature [12]. Finally, unlike progression-prediction systems that require very large, mono-ethnic cohorts with limited generalizability [7], our approach is resource-aware (T4 GPU), reproducible, and fully reported, making it suitable for broader adoption.

The preprocessing and detection methods employed by researchers to classify and identify various eye disorders are comprehensively summarized in this review of the literature. Researchers frequently combine different feature types to achieve more accurate classification and detection owing to the impact of these features on disease prediction. Improving the quality of a dataset is an important step that affects prediction performance. If we survey past research, we can easily find that, in most cases, DL is used for detecting cancers, tumors, skin diseases or plant diseases. Most studies were based on various algorithms with different types of datasets. VGG16, ResNet50, DenseNet121, and AlexNet are widely used in diagnosing disorders. Although much research has been conducted on the detection of a disorder, most of it is not fully dedicated to diabetic retinopathy. Therefore, to acquire an understanding of the most current developments in ML based on disease detection, focusing on diabetic retinopathy, the authors reviewed multiple research studies. After implementing various pretrained ML base models with and without hyper-tuning, the authors proposed a DL model, DR-RetinaNet, where the backbone architecture is DenseNet201 with some suitable tuning parameters that will detect DR disorder with higher accuracy. This proposed model will be highly beneficial in the medical industry for detecting DR with much higher accuracy.

3 Research Methodology

Fig. 2
Fig. 2
Full size image

Workflow for detecting diabetic retinopathy using the proposed DR-RetinaNet model: The pipeline starting from the acquisition and preprocessing of the DR dataset is shown in the figure. The data is divided into 80% for training and 20% for validation. Transfer learning was conducted using a range of pre-trained models, namely, AlexNet, DenseNet201, EfficientNetB0 and so on. Finally, the proposed DR-RetinaNet model will be used for classification and detection to provide the output of the disease prediction

An extensive study of the proposed methodology, including the preprocessing stages, is given in this section. The dataset was first obtained for the analysis. Preprocessing procedures then guarantee data set homogeneity; in this study, the data set has been balanced, and some preprocessing methods have been used to enhance image quality. In this study, we reviewed multiple studies focusing on this disease. After a thorough investigation, the authors proposed a model, DR-RetinaNet, of the DenseNet201 backbone after hyper-tuning and adding customized layers.

Figure 2 presents a block diagram of the model. The detailed framework of this methodology is described in the following subsections.

3.1 Environmental Setup

The model was developed using Google Colab with a T4 GPU accelerator for a faster computation. The training process was configured with a learning rate of 0.00001 and the Adam optimizer, known for its efficient handling of sparse gradients and adaptive learning rates. Training was set to 25 epochs but was stopped at the 12th epoch using early stopping on validation loss to avoid overfitting. Batch size has been chosen to be 32 because of a trade-off between memory consumption and stability of gradients.

3.2 Data Collection

The dataset used in this study originates from the APTOS 2019 Blindness Detection Challenge [27], hosted on Kaggle. The original images were collected by Aravind Eye Hospital (India) as part of their initiative to detect and prevent diabetic retinopathy in rural populations where medical screening is difficult to conduct. The dataset has five classes, namely, No_DR, Mild, Moderate, Severe, and Proliferate_DR. These five types need to be understood thoroughly for the research work. In the next section, the authors provide a brief explanation (Figure 3):

No_DR: The full form of No_DR represents “No Diabetic Retinopathy”. As the name suggests, no abnormalities were observed in the fundus images. Fundus images are eye images captured using a fundus camera. However, No_DR indicates the eye images seem to be healthy.

Mild Diabetic Retinopathy: Mild diabetic retinopathy can be identified as the next stage of No_DR. In this stage, some small dot-like or balloon-like spots are seen in the blood vessels that are mainly named microaneurysms. This means that the blood vessels are slightly damaged, but they do not hamper eyesight.

Fig. 3
Fig. 3
Full size image

Sample images of the DR dataset: The figure shows sample retinal images for five categories of diabetic retinopathy, namely: No_DR, which means no diabetic retinopathy; Mild; Moderate; Proliferative_DR; and Severe

Moderate Diabetic Retinopathy: As we can understand from the name itself, in this class, the images refer to moderate damage to the eyes. In this stage, severe changes occur and it may cause the blood vessels to be leaked or blocked which can be the reason for eye-sight loss. Leaking of blood vessels is called hemorrhages.

Severe Diabetic Retinopathy: This is considered to be the last and most dangerous stage of diabetic retinopathy. The chances of vision loss and glaucoma were extremely high at this stage.

Proliferative_DR: This is a more advanced stage of DR. Abnormal growth of tissues in the blood vessels is the most common occurrence at this stage. Tissue or blood vessel deformation can lead to scarring of the back of the retina. In this stage, loss of sight is highly common.

3.3 Data Preprocessing

As discussed earlier, the images in the dataset were 224 \(\times\) 224 pixels. We did not perform image resizing. We have put 2929 images for training and 733 images for testing at an 80:20 ratio.

3.3.1 SMOTE to Handle Class Imbalance

ML practitioners frequently face challenges owing to imbalanced datasets. One well-known method for resolving this problem is the Synthetic Minority Oversampling Technique (SMOTE). SMOTE creates synthetic samples for minority classes to alleviate imbalance, helping to reduce overfitting caused by random oversampling. It works by interpolating between existing samples in the feature space to generate new instances.

Table 2 Number of images in each class before applying SMOTE: the dataset shows significant imbalance, with the Severe and Proliferative DR classes heavily underrepresented compared to No_DR and Moderate
Full size table

In our dataset (Table 2), a significant imbalance was observed across the five DR classes. To address this, we employed SMOTE using the imblearn library. Before applying SMOTE, each image was reshaped into a one-dimensional vector of pixel intensities, allowing SMOTE to operate in the feature space by interpolating between minority class samples. While applying SMOTE in raw pixel space is unconventional and may not preserve realistic image structures, this approach was adopted for computational simplicity and yielded improved class balance during training. Importantly, SMOTE was applied only to the training set after the train/validation split, thereby preventing data leakage or performance inflation. We note that feature-level SMOTE, based on embeddings from pre-trained networks, is a more common alternative in medical imaging and may be explored in future work.

3.4 Model Architecture

Fig. 4
Fig. 4
Full size image

Systematic pipeline for diabetic retinopathy classification using modified DenseNet201 architecture: The pipeline consists of four stages: (1) Data acquisition, where DR fundus images are collected; (2) Data preprocessing, including normalization and class balancing; (3) Dataset splitting, with 80% of images used for training and 20% for validation; and (4) Training and classification, where features are extracted through the modified DenseNet201 backbone, followed by classification into five DR grades. Final evaluation metrics include accuracy, precision, recall, F1-score, confusion matrix, and learning curves

Figure 4 demonstrates the basic flow diagram of the DR-RetinaNet. The proposed methodology employs a systematic four-phase approach for diabetic retinopathy classification. Initially, the DR Dataset undergoes Data Acquisition, followed by Data Preprocessing, which incorporates normalization and balancing procedures to optimize data quality. Subsequently, Dataset Splitting partitions the processed images into training (80%) and validation (20%) subsets for model development and evaluation. The Training for Classification phase implements a modified DenseNet201 architecture within the DR-RetinaNet framework, utilizing densely connected hidden layers to extract hierarchical features from retinal images. Model performance is assessed through standard evaluation metrics including accuracy, precision, recall, F1-score, confusion matrix analysis, and learning curve examination. This structured pipeline ensures systematic model development and comprehensive performance evaluation for automated diabetic retinopathy detection.

3.4.1 Modified DenseNet201 Architecture

We used DenseNet201 as the backbone architecture. The architecture outperformed other state-of-the-art models in our experiments.

The acronym DenseNet stands for Densely Connected Convolutional Networks. Figure 5 represents the basic architecture of DenseNet. Two compelling features set DenseNet apart from other CNN architectures. One of the unique aspects that make DenseNet special is its dense block structure, where every layer is feedforwardly connected to every other layer, promoting feature reuse and efficient gradient flow.

DenseNet201, compared to its shallower counterparts like DenseNet121, offers greater depth and enhanced representational power, which is particularly beneficial for detecting subtle and fine-grained pathological features in retinal images. Its deeper architecture allows the network to learn richer and more abstract feature hierarchies, essential for accurate DR detection.

Fig. 5
Fig. 5
Full size image

Basic architecture of DenseNet: The basic architecture of DenseNet, comprises several Dense Blocks connected by convolutional and pooling layers. Each Dense Block has a dense connection that guarantees effective feature reuse. The network ends with a global pooling layer and fully connected layers for classification

Secondly, it makes use of bottleneck layers, which assist in lowering the number of parameters without lowering the total amount of characteristics the network learns [28].

Fig. 6
Fig. 6
Full size image

Custom layer of the proposed architecture upon DenseNet201: This architecture involves customized dense blocks, dropout for regularization, and dense layers for classification. DenseNet201 was used as the backbone for the proposed DR-RetinaNet model. The layers before conv5_block15_0_bn were set to non-trainable, and all subsequent layers were set to trainable. Further, batch normalization layers were frozen to stabilize training and avoid degradation of performance. The last layers consist of dropout for regularization, feature reduction using dense layers, and finally, a classification layer that produces five outputs corresponding to the grades of diabetic retinopathy

Moreover, the dense connectivity pattern ensures strong gradient propagation, mitigating vanishing gradient issues and improving training efficiency even with a very deep network. In preliminary experiments, DenseNet201 consistently delivered superior performance compared to other popular architectures, validating its suitability for our task.

Therefore, we used DenseNet201 as the backbone for our proposed DR-RetinaNet model, refining the original architecture to better suit the specific requirements of DR detection. Figure 6 illustrates the customized layer arrangement of DR-RetinaNet.

We build DR-RetinaNet on top of DenseNet201, pre-trained on ImageNet, as the backbone feature extractor. Input fundus images are resized to 224\(\times\)224 RGB and normalized following ImageNet statistics. The DenseNet stem and dense blocks generate hierarchical convolutional features that form the basis for classification.

To preserve generic low-level representations while adapting higher-level features to diabetic retinopathy, we freeze all layers up to and including conv5_block15_0_bn (Keras DenseNet201 naming) and enable fine-tuning from conv5_block16_0_bn onward. This selective freezing strategy substantially reduces training time and improves stability while ensuring that task-specific representations are learned. In practice, the majority of parameters remain frozen, with only the deepest DenseNet layers and the custom classification head updated during training. Additionally, all Batch Normalization layers are kept in inference mode to avoid instability and overfitting, which are common challenges when transferring ImageNet models to relatively small and heterogeneous medical datasets.

On top of the DenseNet backbone, we add a lightweight custom classification head tailored for five-class DR detection. The final 7\(\times\)7\(\times\)1920 feature maps are flattened into a 94,080-dimensional vector, followed by a dense layer with 128 units and ReLU activation. To improve generalization, a dropout layer with a rate of 0.4 is applied, and finally a dense layer with five units and softmax activation produces the class probabilities. This compact head was deliberately chosen to balance accuracy with efficiency, making the model practical for deployment on modest hardware such as a T4 GPU.

Training was conducted in Google Colab (T4 GPU) using the Adam optimizer with a learning rate of \(10^{-5}\), a batch size of 32, and 25 epochs. Early stopping on validation loss was applied, and the best-performing model was selected at epoch 12.

DenseNet’s dense connectivity promotes feature reuse and strong gradient propagation, mitigating vanishing gradient problems in deep networks. The 201-layer variant offers richer representational capacity compared to shallower alternatives such as DenseNet121, which we found particularly beneficial for capturing subtle and fine-grained retinal lesions. Furthermore, DenseNet’s bottleneck and transition layers effectively control parameter growth while maintaining high accuracy, making it a strong foundation for our proposed DR-RetinaNet.

4 Evaluation Metrics

We used the accuracy, precision, f1 score, learning curves, and confusion matrix to evaluate the model in this research.

Accuracy: Many academics use accuracy or error rate as one of the most frequently used metrics to evaluate the effectiveness of classifier classification [29].

$$\begin{aligned} Accuracy = \frac{TP + TN}{TP + TN + FP + FN} \end{aligned}$$
(1)

Precision: The precision of a module is the ratio of its correct modules to the expected defective modules. Precision is crucial, particularly in cases where class distributions are significantly biased [29].

$$\begin{aligned} Precision = \frac{TP}{TP + FP} \end{aligned}$$
(2)

Recall: The percentage of accurate positive predictions among all possible positive forecasts is known as the recall [29].

$$\begin{aligned} Recall = \frac{TP}{TP + FN} \end{aligned}$$
(3)

F1-score: The F1 score was calculated by taking the average harmonic of the accuracy and recall scores. We can calculate it by adding up the reciprocals of all the numbers in a set and then dividing that total by the number of numbers in the set. [29].

Subsequently, the confusion matrix and learning curves were considered. Moreover, to define the performance of the proposed DR-RetinaNet model, its accuracy will be compared with 19 existing models.

5 Results and Discussion

To evaluate the proposed model, we first trained our model, DR-RetinaNet. This command has several attributes. We used batch sizes of 32 and 25 epochs. In addition, we imported and called the EarlyStopping() method. This method was used to prevent overfitting and stop training if it did not improve in consecutive epochs. Table 3 lists the outputs of some epochs to highlight the results.

After epoch 12, the training was completed because of early stopping. It shows that these measures offer valuable information about the model’s performance during training and its ability to generalize new data. With good validation accuracy, low loss values, and high training accuracy, the model appears to operate well and does not noticeably overfit.

Table 3 Accuracy, loss, validation accuracy and validation loss of some epochs: some selected epochs were highlighted to demonstrate the model’s performance
Full size table

5.1 Learning Curve

Fig. 7
Fig. 7
Full size image

Learning curve of DR-RetinaNet model: the figure shows the learning curves for the DR-RetinaNet model during training. It shows the loss and accuracy of both training and validation. The training loss decreases consistently, showing good learning, while the validation loss stabilizes at a low level, which implies marginal overfitting. Similarly, both the training and validation accuracies increase consistently; this is indicative of good generalization from unseen data

Figure 7 shows the learning curves of DR-RetinaNet across epochs. Training accuracy rises steeply while training loss decreases rapidly in the initial stages, indicating effective learning. Validation loss also drops early, suggesting good generalization on unseen data. As training progresses, validation accuracy stabilizes with slight oscillations, showing the model approaching its generalization capacity. Notably, a clear gap remains between final training accuracy (99%) and validation accuracy (94%), which reflects slight overfitting. While this overfitting is relatively constrained, it indicates scope for further improvement. In future work, we plan to employ stronger regularization strategies, such as higher dropout, weight decay, and label smoothing, alongside richer augmentation schemes to further reduce this gap and enhance robustness on unseen data.

5.2 Confusion Matrix

Fig. 8
Fig. 8
Full size image

Confusion matrix of the proposed model: the figure shows the confusion matrix for the proposed DR-RetinaNet model for five classes of No, Mild, Moderate, Severe, and Proliferate classes. The diagonal elements in this figure represent correct predictions made by a model, while its high values indicate a strong capability of the model. Off-diagonal elements are misclassifications, which are very less, suggesting robustness in identifying different stages of diabetic retinopathy

Figure 8 shows the confusion matrix of the model. The confusion matrix shows the performance of the model on the five classes: No, Mild, Moderate, Severe, and Proliferate DR. The correct predictions given by the model are given in the diagonal, where the true positives are much higher in the “No", “Mild", “Severe", and “Proliferate" classes corresponding to 346, 354, 350, and 345, respectively. In the “Moderate" class, its performance is relatively a bit lower, with 304 correct predictions and some misclassifications into “Mild" and “Severe." Off-diagonal values denote misclassifications, including a few instances where “Mild" samples were predicted as “Moderate" and “Moderate" samples were classified as “Severe". Overall, the model performed well, especially for the “No", “Mild", and “Severe" classes.

The confusion matrix shows most errors in the Moderate class, which are often predicted as Mild or Severe. This is clinically plausible because Moderate DR is a middle stage that naturally shares features with both sides. It may still look close to Mild in some cases, while in others it begins to resemble Severe. As a result, the boundaries are fuzzy, and even trained ophthalmologists often disagree when grading Moderate DR. Lower resolution input (224\(\times\)224), Gaussian filtering, and variable image quality can make these subtle signs harder to distinguish, further increasing confusion. Class imbalance also reduces the model’s exposure to sufficient Moderate cases. In future, we plan to explore lesion-focused auxiliary signals, higher-resolution inputs, and targeted augmentation of Moderate samples to improve reliability in this challenging class.

Table 4 Classification report: the overall performance of the DR-RetinaNet was demonstrated
Full size table

Table 4 also illustrates the high performance with a general accuracy of 94% for five classes. All the classes exhibited very high precisions, recall, and F1-scores, which indicates that the model performance was very good in terms of classifying the samples quite correctly. Among these, the highest mark was obtained by Class 0 (No), with an F1-score of 0.97, indicating almost perfect precision and recall. Class 1 (Mild) obtained an F1-score of 0.94, with a few minor misclassifications from other classes. Class 2 (Moderate) reached 0.84 in the recall, giving it an F1-score of 0.89, which would seem to indicate that Moderate cases are somewhat more problematic in their classification. An F1-score of 0.94 and 0.96 for classes 3 and 4, which represent the Severe and Proliferate classes, respectively, indicate that the model can successfully classify these classes; both classes’ weighted average and macro average were 0.94, indicating consistent model performance across classes.

5.3 Performance Comparisons with the Existing Works

We described the specific performance outcome of the proposed model, DR-RetinaNet, in the performance analysis section. We calculated the precision, recall, F1-score, training, and validation accuracy explicitly. DR-RetinaNet achieved training and validation accuracies of 99% and 94% which are the highest among the already existing models. None of the 19 base models that the authors examined provided excellent accuracy with this dataset. We compared the accuracy, precision, recall, and F1-score. Table 5 shows the values obtained for the 19 base models.

Table 5 Precision, Recall, F1-score, and Accuracy from 19 base models and DR-RetinaNet: in Table 3, the precision, recall, F1-score and validation accuracy of the 19 base models and DR-RetinaNet were demonstrated
Full size table

We now will visualize these data and compare them with our proposed model, DR-RetinaNet.

Fig. 9
Fig. 9
Full size image

Performance comparison between different pre-trained models and the proposed DR-RetinaNet architecture: Figure 8 represents the overall performance of the DR-RetinaNet model in a graph compared with the existing 19 base models. DR-RetinaNet achieved superior performance in precision, F1-score, recall, and accuracy

As summarized in Table 5 and Figure 9, DR-RetinaNet achieves 94% precision, 94% recall, 94% F1-score, and 94% validation accuracy, outperforming all 19 baselines by a wide margin. Baseline validation accuracies span 59–79% (e.g., EfficientNet-B0/V2L at 78%, DenseNet121 at 79%), whereas DR-RetinaNet delivers a +15–35 percentage-point gain in accuracy and a +32–59 pp gain in F1-score over typical baselines. Notably, compared with the same backbone without our adaptations (DenseNet201 row), performance improves from 61% \(\rightarrow\) 94% accuracy (+33 pp) and 43% \(\rightarrow\) 94% F1 (+51 pp), indicating that the proposed head, selective freezing, and BN handling substantially enhance discriminative power. Moreover, several baselines exhibit imbalanced precision/recall (and depressed F1), suggesting inconsistent behavior across classes, while DR-RetinaNet maintains balanced precision and recall (both 94%), evidencing robustness rather than a trade-off between false positives and false negatives.

After comparing our work with the already trained models (Table 4), it is clear that our proposed model will work in the early screening of DR far more accurately than the other already existing models, as in all aspects, our proposed model is gaining much higher values than the pretrained models.

In earlier studies, as enumerated in Table 1, numerous researchers proposed solutions to transcend similar issues. However, the majority of them had certain restrictions, which are attempted to be transcended in this study. For instance, some relied on human interpretation for tagging [8], or were constrained by dataset-related issues, i.e., accuracy is dependent on clinical validation [15] or having imbalanced data distribution [18]. In addition, some models required large datasets to achieve legitimate accuracy [20, 21], while others required enormous computational resources [24, 26]. These research gaps have motivated us to design a system that is independent of dataset size and human labeling. Moreover, the proposed DR-RetinaNet is not reliant on high-performance computing power, hence being more efficient and accessible.

In addition to comparisons with state-of-the-art deep learning models, we considered the performance of DR-RetinaNet relative to existing clinical screening methods. Traditional DR screening typically relies on manual grading of fundus images by ophthalmologists or trained graders, which is time-consuming, subject to inter-grader variability, and dependent on the availability of specialized personnel.

In contrast, DR-RetinaNet consistently achieved a precision, recall, and F1-score of 94%, along with a validation accuracy of 94%, suggesting a more consistent and high-performance alternative. Furthermore, the automation provided by DR-RetinaNet can reduce human workload, minimize subjectivity, and enable faster, scalable DR screening, especially valuable in resource-limited settings.

The results obtained with the proposed model demonstrate impressively high accuracy and validation scores, highlighting its potential to significantly enhance DR detection. Early and accurate identification of DR is crucial because it enables timely clinical interventions, potentially preventing irreversible vision loss in diabetic patients.

In real-world applications, the proposed model could be integrated into primary healthcare centers, community screening programs, or telemedicine platforms, offering a scalable and efficient solution for mass DR screening. By providing automated, reliable assessments, the model can reduce the workload of ophthalmologists, allowing them to focus more on treatment planning rather than initial diagnosis. This would be particularly impactful in regions with limited access to specialized eye care professionals. Furthermore, the model’s high accuracy could help minimize misdiagnosis rates, leading to better patient outcomes and more efficient resource allocation within healthcare systems.

In general, these clinical implications underscore the relevance of the model not only in controlled experimental settings but also in practical, real-world healthcare settings, offering a valuable tool to improve early detection and management of DR.

6 Conclusions

Diabetic retinopathy is increasingly common among diabetic individuals, especially the elderly, and can cause irreversible vision loss if not detected early. In this study, we reviewed prior research and evaluated 19 state-of-the-art models on a chosen dataset, finding that none achieved satisfactory performance. To address this, we proposed DR-RetinaNet, a DenseNet201-based architecture with customized layers and selective fine-tuning. Our model achieved 99% training accuracy and 94% validation accuracy, outperforming existing methods in precision, recall, and F1-score. Given its lightweight structure and strong accuracy, DR-RetinaNet has significant potential for integration into mobile screening tools, enabling real-time DR detection in remote or under-resourced areas.

Future work will directly target the current limitations identified in this study. To address the slight overfitting observed in the learning curves, we will explore stronger regularization techniques (e.g., increased dropout, weight decay, label smoothing) and richer augmentation strategies. To improve generalizability, we plan to validate DR-RetinaNet on larger, multi-center datasets collected across diverse populations and imaging devices. In addition, we will extend the dataset to include other eye diseases, allowing the model to become more robust across related conditions. Finally, we will conduct formal statistical significance testing against baseline models (e.g., McNemar’s test or paired t-tests) to provide more rigorous validation of the proposed approach.