Introduction: Beta-lactam (BL) antibiotics are the most often involved drugs in allergic reactions. Mild cutaneous reactions such as maculopapular exanthema or urticaria are the most common presenting complaints of BL allergy in the pediatric population. However, it can be challenging to distinguish BL-induced allergy from reactions due to infections or other reasons. In this study, we aimed to determine the clinical characteristics and potential risk factors of true BL allergy in children with suspected mild cutaneous reactions to BLs. Methods: We evaluated children who were admitted to our pediatric allergy clinic with suspected BL allergy in between January 2015 and March 2020. Patients with a history suggestive of immediate and non-immediate mild cutaneous reactions were included in the study. The oral challenge test (OCT) with the culprit drug was performed on all patients to confirm the diagnosis. Results: Two hundred fourteen (119 male and 95 female) patients with a median age of 4.9 years were evaluated. BL allergy was confirmed in 10.7% (23) of the patients, according to the OCT results. Most of the proven allergic reactions were of the immediate type (73.9%), and urticaria was the most common presenting complaint (60.8%) in proven BL-allergic patients. The negative predictive value of penicillin-G skin testing was 89.7% for immediate-type penicillin allergy and 93.4% for non-immediate reactions. Also, positive predictive value of penicillin-G skin testing was 50% for immediate and 25% for non-immediate reactions. In the multivariate logistic regression analysis, a history of proven drug allergy (Exp (B): 7.76, 95% CI: 1.88–31.97, p = 0.005) was found to be the risk for BL allergy. Conclusion: This study highlighted that OCTs should be performed to confirm the diagnosis in patients suspected of immediate and non-immediate mild cutaneous reactions to BLs and remove the overestimated “BL allergy” label. In these patients, a history of proven drug allergy might be a risk factor for true BL allergy.