End-To-End Neural Network Based Captcha Recognition

  • Jusin Jusin
  • Wilbert Harriman Universitas Pelita Harapan
  • Robin Robin Universitas Pelita Harapan


Completely Automated Public Turing test to tell Computers and Humans Apart (CAPTCHA) is widely used as a security measure against spam and bot attacks via the Internet. CAPTCHA works by the assumption that it takes human sensory and cognitive skills (that are not present in computers) to successfully identify objects or letters within a noisy graphical environment. In this work, we propose a way to teach machines to recognize CAPTCHAs with deep learning. Our deep learning model uses a Convolutional Neural Network (CNN) encoder to convert CAPTCHA images into vector representations, followed by a Recurrent Neural Network (RNN) decoder to convert vector representations into text. Our model is able to achieve a validation accuracy of 90% after about an hour of training. Code is available at https://github.com/wilbertharriman/tf2-attention-captcha-recognizer.
Jul 28, 2022
How to Cite
JUSIN, Jusin; HARRIMAN, Wilbert; ROBIN, Robin. End-To-End Neural Network Based Captcha Recognition. Journal Information System Development (ISD), [S.l.], v. 7, n. 2, p. 28 - 33, july 2022. ISSN 2528-5114. Available at: <https://ejournal-medan.uph.edu/index.php/isd/article/view/552>. Date accessed: 03 oct. 2022. doi: http://dx.doi.org/10.19166/isd.v7i2.552.